Scott Alexander doesn’t like growth mindset… yet.

I recently read Scott Alexander’s posts about growth mindset (one, two, three, four). He starts out admitting that he’s biased against it, and I agree — reading his criticism made me take growth mindset more seriously, because his criticism was rather weak and he obviously tried pretty hard to knock it down.

Before reading his take, all I knew of growth mindset was that there are a lot of people who go around responding “Yet! Growth mindset!” whenever they hear things like “I’m not good at math” which sound like they imply fixed skill levels. It’s mostly a silly in-joke. You wait for someone to say they “can’t”, and then you pounce: “YET!”

Scott divides growth mindset into two possible versions, the “Sorta Controversial Position” (which he thinks might not be true outside the lab) and the “Very Controversial Position” (which he finds almost absurd). Quoting his second article:

SCP: The more children believe effort matters, and the less they believe innate ability matters, the more successful they will be. This is because every iota of belief they have in effort gives them more incentive to practice. A child who believes innate ability and effort both explain part of the story might think “Well, if I practice I’ll become a little better, but I’ll never be as good as Mozart. So I’ll practice a little but not get my hopes up.” A child who believes only effort matters, and innate ability doesn’t matter at all, might think “If I practice enough, I can become exactly as good as Mozart.” Then she will practice a truly ridiculous amount to try to achieve fame and fortune. This is why growth mindset works.

VCP: Belief in the importance of ability directly saps a child’s good qualities in some complicated psychological way. It is worse than merely believing that success is based on luck, or success is based on skin color, or that success is based on whatever other thing that isn’t effort. It shifts children into a mode where they must protect their claim to genius at all costs, whether that requires lying, cheating, self-sabotaging, or just avoiding intellectual effort entirely. When a fixed mindset child doesn’t practice as much, it’s not because they’ve made a rational calculation about the utility of practice towards achieving success, it’s because they’ve partly or entirely abandoned success as a goal in favor of the goal of trying to convince other people that they’re Smart.

(Fixed mindset is the name for the opposite of growth mindset.)

A researcher studying growth mindset responds to Scott by email (quoted in the fourth article) saying that Scott misunderstands the claim behind growth mindset, and that neither Carol Dweck (the originator of growth mindset research) nor other researchers define growth mindset like that. I’ll talk more about what Carol Dweck says in her book Self Theories, but although I think Scott does get it a little wrong, I think he’s mostly right that she advocates the VCP. I’ve also come to believe the VCP is mostly true!

Scott’s Case Against

Scott’s first argument against growth mindset is the evidence is too damn good. Growth mindset studies have big effect sizes, excellent statistical significance, and replicate well. Scott doesn’t make his reasoning explicit, but I suspect he’s referencing an idea which has emerged from the replication crisis, which is if you see more positive results than you’d expect, that’s evidence of bias. Taking this into account properly can make our confidence decrease as evidence gets stronger. Growth-mindset results replicate well, which means it isn’t likely an illusion created by p-hacking. That leaves publication bias as a possible culprit. This meta-analysis concluded that publication bias could only be creating small distortions. I tentatively conclude that we’re in the clear: growth mindset seems unlikely to be another illusion to be washed away by improved rigor.

Second, Scott complains that it’s just not very plausible: telling kids who fail that they just have to try harder is mean, not inspiring. His wording in the first post against growth mindset suggests that there’s research backing this up, which would indeed be a point against growth mindset; but he links to one of his own posts, and I don’t see any research cited in it. In any case, he argues that blaming fixed biological intelligence for our failures is not only more true, but more compassionate than blaming low effort.

Which brings us to another point. Scott actually doesn’t make as big a deal out of this as I’d expect, but reading Self Theories, I got pretty annoyed at this aspect of the work. Carol Dweck has several ways of inducing growth or fixed mindset, but the biggest one (and the one which she thinks is most important, as I understand it) is to lie to kids about how intelligence works. In some studies she uses fake psychology articles, some of which talk about fabricated research showing that intelligence levels are malleable, while another set of articles (given to the other set of subjects) discusses similarly fake research showing the opposite. The fake research is delivered mixed in with vivid stories from the lives of geniuses which support one side or the other.

Now, Carol Dweck doesn’t outright deny any of the research linking intelligence to biology or measuring its stability across a lifetime. Scott states in strong terms that he isn’t accusing her of that. However, I think it’s pretty clear that she would have you believe such research is wrong. In terms Scott might prefer, it’s a motte-bailey ideology. She tells other researchers that growth mindset isn’t claiming biology doesn’t matter for intelligence; rather, it’s claiming that if you believe biology doesn’t matter, you do better. This allows her to advocate telling people that intelligence is malleable, in no uncertain terms, while not needing to claim that intelligence is indeed malleable.

The nature of the game becomes really clear in chapter 9, her chapter on IQ. She can’t bring herself to look at the research on IQ (which is quite unfortunate, since there are really interesting issues here — I’ll address that later). Instead, she side-steps the research by citing what the inventor of IQ thought, saying that he “knew” IQ wasn’t a measure of fixed potential. She gives a few anti-IQ citations, but without saying what any of them accomplish; looking at them, most don’t seem too promising.* She goes on to raise various doubts about how intelligence should be defined. She cites her own research showing that growth-mindset individuals (who are sorted out from the rest via a survey which ensures their views fit her own) define intelligence by phrases like “how much effort you put in and your willingness to learn and do all that you can to fully understand it”. Finally, she ends the chapter by admitting she’s not very interested in the truth of the matter:

The goal of this book is not really to resolve what intelligence is, but rather to ask: What is the most useful way of thinking about intelligence and what are the consequences of adopting one view over another? I think our research findings speak very clearly to this issue.

GAaah. :(

Despite my frustration, I could forgive her for that. If this is the reality we’re living in — if people who believe false things about intelligence simply do better that people who believe the truth — her position is arguably the wise one. She should be beating around the bush when it comes to IQ research, because which view is more useful is more important than which is true. This way of thinking annoys me terribly, but it’s not really a point against Carol Dweck’s research.

However, it gets worse for growth mindset.

I think this is Scott’s most effective argument against growth mindset: every study shows growth-mindset individuals starting out at an equal academic performance level to fixed-mindset individuals (or lower!). This leads him to say:

The studies don’t show any real-life correlation between growth mindset and any measures of success.

This isn’t quite true; in her book, Dweck discusses a couple of studies which show correlation with later success. But that only makes it even more puzzling! Dweck herself emphasises, in study after study discussed in Self Theories: when students are tested for growth mindset vs fixed mindset, the groups are almost always equal in academic performance starting out. Nonetheless, growth-mindset students perform better in Dweck’s experiments.

One possibility is that growth mindset doesn’t show any differences in performance in the grade-school students that Dweck mainly studies, but creates a widening gap later. In chapter 5 of self-theories, Dweck discusses a study which shows this happening in the transition from grade school to junior high. Fixed-mindset students declined in standing across this transition; especially the fixed-mindset students with the best grades starting out. Similarly, growth-mindset students with the worst grades starting out showed the most improvement. I don’t know whether there was a difference in average grades between growth-mindset and fixed-mindset students in higher grades. She doesn’t say in the book, and there wouldn’t have to be: mindsets could change over time in a way that destroyed the correlation with GPA at any one grade. I’d expect her to mention a difference emerging if it had happened, since a growing gap between the GPA of growth-mindset kids and fixed-mindset kids would speak in favor of growth mindset.

A different study (also discussed in chapter 5) looked at college students. In that case, the fixed-mindset students entered college with slightly better SAT scores. They then did as well as growth-mindset students, meaning that they underperformed relative to their SAT scores. Keep in mind that since colleges are trying to select the best students, this doesn’t really suggest the growth-mindset students started out academically worse; perhaps their applications made up for it in other ways. (The college studied was Berkeley, so standards were high.) In that case, we’d expect low-SAT students to over-perform relative to their SAT alone, and high-SAT students to under-perform. doesn’t seem too convincing as an example of growth-mindset doing better. However, I see no indication of that in the study — growth-mindset and fixed-mindset groups were the same as each other in high-school GPA on entry. (High-school GPA was the only other on-entry academic ability statistic they looked at.) So, it’s hard to say. At least we can conclude that growth mindset didn’t make up for the initial differences in academic performance between the groups.

Overall, it looks like the case for long-term benefits is weak in the book. There are some results I haven’t mentioned yet, which do show long-term benefits for minority students. The study Scott Alexander spent so much time complaining about was like that, too. But I have to wonder: if there wasn’t an effect on average, but there was a positive effect for minorities, doesn’t that mean it’s a negative effect for non-minorities?

All this seems like a fairly damning case against the importance of growth mindset. I’ll explain why I think not as we go on.

But first, what is growth mindset, according to Carol Dweck?

Dweck’s Case For

After reading Scott’s series of posts, I had my suspicions that growth mindset was actually a pretty good hypothesis that summarized a lot of ideas I cared about and put them on a firmer experimental footing. However, I expected to be annoyed by Dweck lumping all these important things under one label, “growth mindset”. I was wrong on that count: in Self Theories, she examines a number of distinct phenomena and doesn’t lump them together under one umbrella even after she’s shown that they correlate with each other (and moreover, establishes cause-effect relations).

Wanting to look good vs wanting to learn. “Wanting to learn” means focusing on the material itself, being driven by challenges, and choosing those things over external validation when they conflict. “Wanting to look good” means focusing on grades, pleasing teachers or parents, or impressing fellow students. Basically, external validation. Carol Dweck uses the terms “performance goals” vs “learning goals” for this distinction, but some psychologists use “performance goals” to mean the exact opposite thing, so I’ll just call these students “validation-oriented”.

Believing intelligence is fixed vs believing it is malleable. She uses the terms “entity theory of intelligence” vs “incremental theory”, but once again I’ll opt out of the terminology. (“Entity theory”?)

Carol Dweck’s experiments show that students who focus on learning handle challenges much better that those who focus on external validation. When students are sorted based on these goals (by taking a survey), the two groups initially perform at the same level when presented with typical problems for their grade (as we bemoaned above). However, when she presents them with problems above their grade level, the two groups show dramatic differences. Learning-oriented students step up their game. They are happy for the challenge and say so. On the other hand, validation-oriented students become frustrated. They give up easily, call themselves stupid, and engage in self-handicapping behaviors. When the students return to problems of an ordinary difficulty level, validation oriented students drop in ability, deflated by their defeat. Learning oriented students perform at their normal levels. After the exercise, validation-oriented students overestimate the number of problems they got wrong, while learning-oriented students remember correctly.

Carol Dweck calls these response profiles helpless response vs mastery response. The term “helpless” invokes research in learned helplessness. Learned helplessness is the phenomenon of learning that you can’t improve a situation, and stopping any effort to do so even when the situation changes. Learned helplessness has been connected to depression; and indeed, research has also found links between fixed mindset and depressive behavior. But why would there be a link between learned helplessness and being validation-oriented?

That’s where a fixed theory of intelligence comes in. If you think that people are either smart or dumb, then it makes sense to conclude that a difficult problem is just too hard far you. It makes sense to then focus on getting external approval: even learning oriented students tend to want that; it’s just not as much of a focus. Experiments show that these traits are linked, and that causing students to have a malleable theory of intelligence or a fixed theory (by showing them the fake research mentioned earlier, or by similar methods) does cause them to have learning goals or validation goals (and the mastery or helpless responses to go with ‘em).

Growth mindset, then, is the combination of a malleable theory of intelligence, learning-oriented goals, and a mastery response to challenges. These three things are correlated, and causally related as well.

All of this also helps to explain the mystery from the previous section, too. Growth mindset only makes a difference in the face of challenges. Carol Dweck gave students problems inappropriate for their grade levels. Teachers work really hard to ensure that this does not happen in the classroom. The way things work, teachers more or less have to cater to the worst students in order to be sure almost everyone can keep up. This is great for fixed-mindset students, who want to be able to get good grades predictably. It’s much worse for the growth-mindset students who seek a challenge. (Sal Khan thinks he can change this, and uses Carol Dwek’s mindset terminology.)

The exception to this rule is the treatment of minority students, who often face language barriers (as in Los Angeles, where children who speak spanish at home go to be taught in english in the public schools) or other difficulties. So, it makes a lot of sense that growth mindset would be more important there.

I did find a study which shows the overall long-term effect in the transition to junior high, though! There’s a nice graph:

I don’t know if that kind of thing replicates or just happened in this particular study, but it’s encouraging. Notice that they only look at math achievement. Of this, they say:

Mathematics is a subject that many students find difficult; thus, it meets the requirement of being a sufficiently challenging subject to trigger the distinctive motivational patterns related to theory of intelligence, which may not manifest themselves in situations of low challenge.

They even go and do a causal model:

How pretty. Well, I’m sold.

VCP is true. Your theory of whether intelligence is fixed or malleable is a root cause for a host of other variables determining outcomes which we care about.

As for the college study which found such unimpressive long-term trends, it could be that grades don’t represent the differences in outcomes very well. If validation-oriented students are choosing easier classes which they know they can get good grades in, and learning-oriented students choose tougher courses which they learn more from, that could easily make up the difference.

I’m not really very certain of all of this. There are a lot of papers that I’ve only read summaries of, and summaries can be very misleading (especially plain-english summaries of statistics). To be really sure, I’d want to actually read each of the studies glossed in Self Theories, looking for alternative explanations. But the truth is, while Scott Alexander is biased against growth mindset, I’m biased for it. Aside from the annoying perspective on IQ, it just seems to get a lot of things right. It’s “in the same spirit” as other things I agree with, such as this.

Let’s see if I can make it as obvious to you as it is to me.

Re-Framing Growth Mindset

Realistic Views of Intelligence

My #1 annoyance with Self Theories is, as I’ve said, the way it advocates “useful” theories of intelligence over factual ones. This seems entirely unnecessary to me. We can encourage growth mindset without over-selling the malleability of intelligence.

First, some facts. How malleable is intelligence? It’s a complicated issue. Within a single generation, variation in adult IQ is about 75% genetics. Much of the rest is made up of other biological factors, such as nutrition and disease (but I haven’t found numbers on that). However, this kind of percentage can be misleading. The Flynn Effect observes that IQ has risen about 2.93 points per decade. This is high enough to contradict a primarily biological explanation for the IQ variation; biological factors vary, but not that much. So, as I understand it, the prevailing theory is that variation within a generation are primarily biological, but variations across generation show a large cultural effect from things like improved schooling.

The Flynn effect may be a point in favor of malleability, but not within-generation. Looking at that alone, we might conclude that non-biological factors of IQ have to be pervasive cultural elements to show up. There do seem to be more short-term factors, though. This study finds that effort accounts for 0.64 standard deviations on IQ tests! This wouldn’t show up as a factor ordinarily, since the experiment specifically incentivised some participants to do well, and not others; something you’d ordinarily never consider when trying to get a good objective measure of IQ.

So, it appears that there is some evidence for malleability. Even so, this is far from the fabricated stories which were used to induce growth mindset in experiments. Do we need to exaggerate the malleability of intelligence?

I’ve held to Carol Dweck’s usage of the term “intelligence” as it relates to growth mindset, in order to illustrate it to you so we can be annoyed at it together. Scott Alexander doesn’t do her the courtesy. Notice that is version of the SCP and VCP don’t mention intelligence. In Carol Dweck’s work, it’s children’s beliefs about intelligence that matter the most. In Scott Alexander’s discussion, he’s talking about how much effort matters in comparison to innate ability. In one case (discussed in chapter 5 of Self Theories), she gives students this equation to see how they fill in the blanks:

Intelligence = _____% effort + _____% ability.

Scott, on the other hand, just talks about how much“effort vs ability” matters. Scott’s version makes more sense. Even if intelligence is a relatively fixed trait, other factors may be just as important to success. Before investigating, I thought that IQ swamped other psychological correlates of success, such as personality traits. Not so. This meta-analysis finds that personality, IQ, and socioeconomic status are about equally responsible for lifetime income. Shouldn’t we be talking about factors of success more broadly? Why is Carol Dweck talking about whether intelligence is fixed?

This is not a battle she needs to be fighting; especially if you take a close look at the causal graph I cited from one of her papers earlier. “Incremental Theory” (the belief that intelligence is malleable) looks like the source of everything in the network, but it is the furthest node from the things we actually care about. Can’t we try other things than modifying the beliefs about intelligence?

I should mention that Carol Dweck did investigate other ways. Simply telling students that a set of problems is for the purpose of evaluating their ability creates validation goals, and a more helpless response when challenges come along. On the other hand, presenting the same problem set as a learning experience, for the purpose of students testing themselves and seeing what they need to work on, creates learning goals and mastery responses toward challenges.

Consulting our handy causal diagram, it looks like “positive effort beliefs” are a really good thing to try and modify. And, to me, this version makes so much more sense. You don’t have to believe something like

Results = 90 % effort + 10 % ability

in order to believe it’s worth putting in lots of effort. Maybe results are 99% due to innate ability. We can still benefit from putting in effort, applying beneficial strategies, focusing on learning goals, and avoiding learned helplessness. You just have to believe more effort pays off.

I’m still saying VCP is essentially true, but we probably don’t have to lie to kids to reap the rewards of growth mindset. Sure, some people have a much higher IQ than you do, and maybe they’ll be better than you at whatever they try. So what? Effort pays off. The amount you’ll eventually be able to do if you dig into challenges is far larger than if you avoid them. The best way to look smart in the long term is to be willing to look dumb in the short term by asking questions, seeking out challenges, and trying things that might fail (so that you can learn). Those things seem so clear to me that I would be rather puzzled indeed if such behavior didn’t have long-term benefits!

Errors vs Bugs

I can imagine Carol Dweck responding: “We do tell subjects that effort pays off, but when we’re trying to install growth mindset, we do everything we can. Leaving out the malleability of intelligence would weaken the effect, since those beliefs drive others.”

Well, I’ve got another proposal for you. Fixed-vs-malleable is just one dimension of how we think about skill. Believing in malleability of skill seems to help kids believe that effort matters, but maybe it’s not the best way. It’s untested, but I think errors vs bugs and the end of stupidity might be a whole lot better. Quoting:

I wasn’t an exceptional pianist, and when I’d play my nocturne for him, there would be a few clinkers. I apologized — I was embarrassed to be wasting his time. But he never seem to judge me for my mistakes. Instead, he’d try to fix them with me: repeating a three-note phrase, differently each time, trying to get me to unlearn a hand position or habitual movement pattern that was systematically sending my fingers to wrong notes.

I had never thought about wrong notes that way. I had thought that wrong notes came from being “bad at piano” or “not practicing hard enough,” and if you practiced harder the clinkers would go away. But that’s a myth.

In fact, wrong notes always have a cause. An immediate physical cause. (more)

Even if we believe that our skill is malleable, we may not think that we have very much power to change it. Perhaps we think it’s tied to some aspect of the environment, like how good the teacher is or whether we get enough intellectual stimulation. But the idea that each mistake has a specific cause is very empowering. Not only is skill changeable; it’s got moving parts that you can examine! This concept also encourages you to work smarter. Blindly practicing, practicing, practicing might be better than not, but directing effort intelligently to figure out what’s missing from your skill and how to correct it will be much more effective.

Time Preference

You might be thinking that my version is just going to lie to the kids at a different point: rather than installing unfounded beliefs in the malleability of intelligence, I’m now in a position where I’d have to install unrealistically high beliefs in the results of effort.

I don’t think this is true. I think the helpless-response group really are sabotaging themselves. They’re failing to put in effort that really would benefit them. In challenging situations, learning-oriented students are going to constantly be one step ahead of validation-oriented students; they’re still dealing with the status implications as learning-oriented students dig in. Validation-students are going to be more past-oriented, focusing on how well they’ve done so far and how to damage-control for failures. Learning-oriented students have all their thoughts on the problem at hand. The result is that validation-oriented students learn less and don’t look as good.

Why would this happen? How is it that the validation-oriented strategies deprive them of the very thing they want? Well, first, validation-oriented students will avoid this type of situation in the first place. By seeking areas where they can already excel, they achieve their validation goals reliably. But second, I think validation-orientation naturally creates short-term thinking. If you’re focusing on grades rather than learning, you’ll tend to study the night before the test. If you’re focused on getting someone’s approval rather than getting things done, you’re likely to make big promises that you won’t be able to keep. This is called high time preference: validation-oriented students prefer immediate reward at the expense of long-term reward.

I’m guessing. I don’t know if there have been any studies associating growth mindset with time preference; I haven’t found them. However, there has been some work in modifying time preference. This study suggests that concrete thinking creates a high time preference, while abstract thinking creates low time preference. If so, it may be that focusing on reactions of people around you creates high time preference, while focusing on intellectual questions associated with learning creates low time preference. This model suggests that time preference is a function of what options you have in mind. Thinking of more near-term events naturally creates a high time preference, while thinking of far-off events creates a low one. If you’re thinking about what grades you will get in upcoming assignments and tests, maybe this creates a high time preference in comparison to thinking less about those things.

Fostering Curiosity

You can change your response to a situation by thinking about a different aspect of the situation. What you focus on is what you react to. In Surely You’re Joking, Mr. Feynman!, Feynman mentions again and again that a big advantage in his life has been that when he’s thinking of physics, he doesn’t have the attention to think about who he is talking to. This means he doesn’t think twice about saying “you’re wrong” to important people, where others would say “yes, very interesting”. This is very close to the distinction between learning-oriented goals and validation-oriented goals.

In my experience, a lot of people get tripped up because they don’t ask “stupid questions” or “make stupid comments” — they assume everyone else knows the answers, or that what’s obvious to them is equally obvious to everyone. These comments or questions are usually much more important than they seem to the one voicing them. Focusing on what you know and what you don’t know is therefore very helpful. Even if you are misguided, this approach is the fastest way to get corrected.

Actually Try

I think the most valuable advice provided by growth mindset is to actually try to solve your problems. That’s the point of the “Yet! Growth mindset!” game which I mentioned at the beginning: reminding people to try. Why would reminding people to try be useful? Can people just not think of trying to solve their problems? I think the answer is yes. This is a little boggling, though. Why would this be?

Part of my answer is that mentioning the possibility of changing things re-frames the time-preference, as mentioned in the previous section. This isn’t the whole story, though. A second piece of the puzzle is that we are finite creatures, and sometimes it just doesn’t occur to us that a particular fact about our situation may be changeable. Reminding ourselves/others to actually try at least raises the possibility to our attention so that we can evaluate it.

A third reason why this might be useful is that we spend a lot of our time only signalling trying. It’s usually enough to put on a good show of effort. Maybe this kind of behavior is useful so often that we don’t frequently consider doing the other kind of thing, where we break out of our social roles and do our damndest to get what we’re after.

This is related to my internal lawyer idea: a lot of human actions are more optimized for justifiability than outcome.

Denouement

If you’re frustrated with the way “growth mindset” lumps a lot of ideas together, I can sympathize. All the traits have been found to correlate positively, so we can use an error model as a convenient approximation. The growth mindset literature is really about a bug model, though; it provides specific cause-and-effect relationships between the parts which have been borne out by the data (and adapted with the data over time). I hope you can get a good idea of many of the pieces and why they seem plausible individually from this post.

The growth-mindset research really seems quite good. In other areas of psychology I’ve tried to dive into in this way, things seem to get murkier and murkier as I chase down data and citations. Here, the picture seemed to get clearer and clearer instead. That’s quite an accomplishment.

*: A book, On Intelligence…More Or Less, looks relevant; but I haven’t looked at it too much yet.