http://www.talyarkoni.org/blog/2010/07/07/What-the-Dunning-Kruger-effect-Is-and-Isnt/ Skip to content [citation needed] and after the citation, some ice cream would be nice Menu and widgets Search for: [ ] [Search] subscribe to this blog via email Enter your email address to subscribe to this blog and receive notifications of new posts by email. Email Address [ ] Subscribe Recent Posts * hey, I wrote a book! * Tropic of Zamza * Induction is not optional (if you're using inferential statistics): reply to Lakens * Fly less, give more * The parable of the three districts: A projective test for psychologists Recent Comments * DUNNING-KRUGER EFFECT: IGNORANCE AND OVERCONFIDENCE AFFECT INTUITIVE THINKING, NEW STUDY SAYS - Gianni Bellorio on what the Dunning-Kruger effect is and isn't * What the Dunning-Kruger effect is and isn't - READING LIST on in defense of three of my favorite sayings * "Why We Sleep" Is Riddled with Scientific and Factual Errors - READING LIST on The Great Minds Journal Club discusses Westfall & Yarkoni (2016) * T R Shankar Raman on Why I still won't review for or publish with Elsevier-and think you shouldn't either * Boris Barbour on No, it's not The Incentives--it's you Archives * November 2020 * July 2020 * May 2020 * January 2020 * September 2019 * July 2019 * October 2018 * August 2018 * November 2017 * October 2017 * May 2017 * December 2016 * October 2016 * June 2016 * April 2016 * December 2015 * May 2015 * March 2015 * February 2015 * January 2015 * October 2014 * July 2014 * June 2014 * May 2014 * April 2014 * March 2014 * December 2013 * November 2013 * September 2013 * August 2013 * July 2013 * April 2013 * March 2013 * February 2013 * June 2012 * May 2012 * April 2012 * March 2012 * February 2012 * January 2012 * December 2011 * November 2011 * October 2011 * September 2011 * August 2011 * July 2011 * May 2011 * April 2011 * February 2011 * January 2011 * December 2010 * October 2010 * September 2010 * August 2010 * July 2010 * June 2010 * May 2010 * April 2010 * March 2010 * February 2010 * January 2010 * December 2009 * November 2009 * October 2009 Tags * academics * art * blogging * boulder * CNS * cognitive neuroscience * data * data mining * data science * depression * effect sizes * ethics * evolution * fiction * fmri * genetic algorithms * incentives * language * measurement * methods * neuroscience * neurosynth * news * okcupid * open science * peer review * personality * power * psychoinformatics * psychology * psychometrics * publishing * python * R * reliability * replication * research * reverse inference * science * statistics * tenure track * tools * training * twitter * writing what the Dunning-Kruger effect is and isn't If you regularly read cognitive science or psychology blogs (or even just the lowly New York Times!), you've probably heard of something called the Dunning-Kruger effect. The Dunning-Kruger effect refers to the seemingly pervasive tendency of poor performers to overestimate their abilities relative to other people-and, to a lesser extent, for high performers to underestimate their abilities. The explanation for this, according to Kruger and Dunning, who first reported the effect in an extremely influential 1999 article in the Journal of Personality and Social Psychology, is that incompetent people by lack the skills they'd need in order to be able to distinguish good performers from bad performers: ...people who lack the knowledge or wisdom to perform well are often unaware of this fact. We attribute this lack of awareness to a deficit in metacognitive skill. That is, the same incompetence that leads them to make wrong choices also deprives them of the savvy necessary to recognize competence, be it their own or anyone else's. For reasons I'm not really clear on, the Dunning-Kruger effect seems to be experiencing something of a renaissance over the past few months; it's everywhere in the blogosphere and media. For instance, here are just a few alleged Dunning-Krugerisms from the past few weeks: ...So what does this mean in business? Well, it's all over the place. Even the title of Dunning and Kruger's paper, the part about inflated self-assessments, reminds me of a truism that was pointed out by a supervisor early in my career: The best employees will invariably be the hardest on themselves in self-evaluations, while the lowest performers can be counted on to think they are doing excellent work... ...Heidi Montag and Spencer Pratt are great examples of the Dunning-Kruger effect. A whole industry of assholes are making a living off of encouraging two attractive yet untalented people they are actually genius auteurs. The bubble around them is so thick, they may never escape it. At this point, all of America (at least those who know who they are), is in on the joke - yet the two people in the center of this tragedy are completely unaware... ...Not so fast there -- the Dunning-Kruger effect comes into play here. People in the United States do not have a high level of understanding of evolution, and this survey did not measure actual competence. I've found that the people most likely to declare that they have a thorough knowledge of evolution are the creationists...but that a brief conversation is always sufficient to discover that all they've really got is a confused welter of misinformation... As you can see, the findings reported by Kruger and Dunning are often interpreted to suggest that the less competent people are, the more competent they think they are. People who perform worst at a task tend to think they're god's gift to said task, and the people who can actually do said task often display excessive modesty. I suspect we find this sort of explanation compelling because it appeals to our implicit just-world theories: we'd like to believe that people who obnoxiously proclaim their excellence at X, Y, and Z must really not be so very good at X, Y, and Z at all, and must be (over)compensating for some actual deficiency; it's much less pleasant to imagine that people who go around shoving their (alleged) superiority in our faces might really be better than us at what they do. Unfortunately, Kruger and Dunning never actually provided any support for this type of just-world view; their studies categorically didn't show that incompetent people are more confident or arrogant than competent people. What they did show is this: [dunning_kruger][dunning_kruger] This is one of the key figures from Kruger and Dunning's 1999 paper (and the basic effect has been replicated many times since). The critical point to note is that there's a clear positive correlation between actual performance (gray line) and perceived performance (black line): the people in the top quartile for actual performance think they perform better than the people in the second quartile, who in turn think they perform better than the people in the third quartile, and so on. So the bias is definitively not that incompetent people think they're better than competent people. Rather, it's that incompetent people think they're much better than they actually are. But they typically still don't think they're quite as good as people who, you know, actually are good. (It's important to note that Dunning and Kruger never claimed to show that the unskilled think they're better than the skilled; that's just the way the finding is often interpreted by others.) That said, it's clear that there is a very large discrepancy between the way incompetent people actually perform and the way they perceive their own performance level, whereas the discrepancy is much smaller for highly competent individuals. So the big question is why. Kruger and Dunning's explanation, as I mentioned above, is that incompetent people lack the skills they'd need in order to know they're incompetent. For example, if you're not very good at learning languages, it might be hard for you to tell that you're not very good, because the very skills that you'd need in order to distinguish someone who's good from someone who's not are the ones you lack. If you can't hear the distinction between two different phonemes, how could you ever know who has native-like pronunciation ability and who doesn't? If you don't understand very many words in another language, how can you evaluate the size of your own vocabulary in relation to other people's? This appeal to people's meta-cognitive abilities (i.e., their knowledge about their knowledge) has some intuitive plausibility, and Kruger, Dunning and their colleagues have provided quite a bit of evidence for it over the past decade. That said, it's by no means the only explanation around; over the past few years, a fairly sizeable literature criticizing or extending Kruger and Dunning's work has developed. I'll mention just three plausible (and mutually compatible) alternative accounts people have proposed (but there are others!) 1. Regression toward the mean. Probably the most common criticism of the Dunning-Kruger effect is that it simply reflects regression to the mean-that is, it's a statistical artifact. Regression to the mean refers to the fact that any time you select a group of individuals based on some criterion, and then measure the standing of those individuals on some other dimension, performance levels will tend to shift (or regress) toward the mean level. It's a notoriously underappreciated problem, and probably explains many, many phenomena that people have tried to interpret substantively. For instance, in placebo-controlled clinical trials of SSRIs, depressed people tend to get better in both the drug and placebo conditions. Some of this is undoubtedly due to the placebo effect, but much of it is probably also due to what's often referred to as "natural history". Depression, like most things, tends to be cyclical: people get better or worse better over time, often for no apparent rhyme or reason. But since people tend to seek help (and sign up for drug trials) primarily when they're doing particularly badly, it follows that most people would get better to some extent even without any treatment. That's regression to the mean (the Wikipedia entry has other nice examples-for example, the famous Sports Illustrated Cover Jinx). In the context of the Dunning-Kruger effect, the argument is that incompetent people simply regress toward the mean when you ask them to evaluate their own performance. Since perceived performance is influenced not only by actual performance, but also by many other factors (e.g., one's personality, meta-cognitive ability, measurement error, etc.), it follows that, on average, people with extreme levels of actual performance won't be quite as extreme in terms of their perception of their performance. So, much of the Dunning-Kruger effect arguably doesn't need to be explained at all, and in fact, it would be quite surprising if you didn't see a pattern of results that looks at least somewhat like the figure above. 2. Regression to the mean plus better-than-average. Having said that, it's clear that regression to the mean can't explain everything about the Dunning-Kruger effect. One problem is that it doesn't explain why the effect is greater at the low end than at the high end. That is, incompetent people tend to overestimate their performance to a much greater extent than competent people underestimate their performance. This asymmetry can't be explained solely by regression to the mean. It can, however, be explained by a combination of RTM and a "better-than-average" (or self-enhancement) heuristic which says that, in general, most people have a tendency to view themselves excessively positively. This two-pronged explanation was proposed by Krueger and Mueller in a 2002 study (note that Krueger and Kruger are different people!), who argued that poor performers suffer from a double whammy: not only do their perceptions of their own performance regress toward the mean, but those perceptions are also further inflated by the self-enhancement bias. In contrast, for high performers, these two effects largely balance each other out: regression to the mean causes high performers to underestimate their performance, but to some extent that underestimation is offset by the self-enhancement bias. As a result, it looks as though high performers make more accurate judgments than low performers, when in reality the high performers are just lucky to be where they are in the distribution. 3. The instrumental role of task difficulty. Consistent with the notion that the Dunning-Kruger effect is at least partly a statistical artifact, some studies have shown that the asymmetry reported by Kruger and Dunning (i.e., the smaller discrepancy for high performers than for low performers) actually goes away, and even reverses, when the ability tests given to participants are very difficult. For instance, Burson and colleagues (2006), writing in JPSP, showed that when University of Chicago undergraduates were asked moderately difficult trivia questions about their university, the subjects who performed best were just as poorly calibrated as the people who performed worst, in the sense that their estimates of how well they did relative to other people were wildly inaccurate. Here's what that looks like: [burson_figure][burson_figure] Notice that this finding wasn't anomalous with respect to the Kruger and Dunning findings; when participants were given easier trivia (the diamond-studded line), Burson et al observed the standard pattern, with poor performers seemingly showing worse calibration. Simply knocking about 10% off the accuracy rate on the trivia questions was enough to induce a large shift in the relative mismatch between perceptions of ability and actual ability. Burson et al then went on to replicate this pattern in two additional studies involving a number of different judgments and tasks, so this result isn't specific to trivia questions. In fact, in the later studies, Burson et al showed that when the task was really difficult, poor performers were actually considerably better calibrated than high performers. Looking at the figure above, it's not hard to see why this would be. Since the slope of the line tends to be pretty constant in these types of experiments, any change in mean performance levels (i.e., a shift in intercept on the y-axis) will necessarily result in a larger difference between actual and perceived performance at the high end. Conversely, if you raise the line, you maximize the difference between actual and perceived performance at the lower end. To get an intuitive sense of what's happening here, just think of it this way: if you're performing a very difficult task, you're probably going to find the experience subjectively demanding even if you're at the high end relative to other people. Since people's judgments about their own relative standing depends to a substantial extent on their subjective perception of their own performance (i.e., you use your sense of how easy a task was as a proxy of how good you must be at it), high performers are going to end up systematically underestimating how well they did. When a task is difficult, most people assume they must have done relatively poorly compared to other people. Conversely, when a task is relatively easy (and the tasks Dunning and Kruger studied were on the easier side), most people assume they must be pretty good compared to others. As a result, it's going to look like the people who perform well are well-calibrated when the task is easy and poorly-calibrated when the task is difficult; less competent people are going to show exactly the opposite pattern. And note that this doesn't require us to assume any relationship between actual performance and perceived performance. You would expect to get the Dunning-Kruger effect for easy tasks even if there was exactly zero correlation between how good people actually are at something and how good they think they are. Here's how Burson et al summarized their findings: Our studies replicate, eliminate, or reverse the association between task performance and judgment accuracy reported by Kruger and Dunning (1999) as a function of task difficulty. On easy tasks, where there is a positive bias, the best performers are also the most accurate in estimating their standing, but on difficult tasks, where there is a negative bias, the worst performers are the most accurate. This pattern is consistent with a combination of noisy estimates and overall bias, with no need to invoke differences in metacognitive abilities. In this regard, our findings support Krueger and Mueller's (2002) reinterpretation of Kruger and Dunning's (1999) findings. An association between task-related skills and metacognitive insight may indeed exist, and later we offer some suggestions for ways to test for it. However, our analyses indicate that the primary drivers of errors in judging relative standing are general inaccuracy and overall biases tied to task difficulty. Thus, it is important to know more about those sources of error in order to better understand and ameliorate them. What should we conclude from these (and other) studies? I think the jury's still out to some extent, but at minimum, I think it's clear that much of the Dunning-Kruger effect reflects either statistical artifact (regression to the mean), or much more general cognitive biases (the tendency to self-enhance and/or to use one's subjective experience as a guide to one's standing in relation to others). This doesn't mean that the meta-cognitive explanation preferred by Dunning, Kruger and colleagues can't hold in some situations; it very well may be that in some cases, and to some extent, people's lack of skill is really what prevents them from accurately determining their standing in relation to others. But I think our default position should be to prefer the alternative explanations I've discussed above, because they're (a) simpler, (b) more general (they explain lots of other phenomena), and (c) necessary (frankly, it'd be amazing if regression to the mean didn't explain at least part of the effect!). We should also try to be aware of another very powerful cognitive bias whenever we use the Dunning-Kruger effect to explain the people or situations around us-namely, confirmation bias. If you believe that incompetent people don't know enough to know they're incompetent, it's not hard to find anecdotal evidence for that; after all, we all know people who are both arrogant and not very good at what they do. But if you stop to look for it, it's probably also not hard to find disconfirming evidence. After all, there are clearly plenty of people who are good at what they do, but not nearly as good as they think they are (i.e., they're above average, and still totally miscalibrated in the positive direction). Just like there are plenty of people who are lousy at what they do and recognize their limitations (e.g., I don't need to be a great runner in order to be able to tell that I'm not a great runner-I'm perfectly well aware that I have terrible endurance, precisely because I can't finish runs that most other runners find trivial!). But the plural of anecdote is not data, and the data appear to be equivocal. Next time you're inclined to chalk your obnoxious co-worker's delusions of grandeur down to the Dunning-Kruger effect, consider the possibility that your co-worker's simply a jerk-no meta-cognitive incompetence necessary. ResearchBlogging.orgResearchBlogging.orgKruger J, & Dunning D (1999). Unskilled and unaware of it: how difficulties in recognizing one's own incompetence lead to inflated self-assessments. Journal of personality and social psychology, 77 (6), 1121-34 PMID: 10626367 [g_100x25_D][g_100x25_D] Krueger J, & Mueller RA (2002). Unskilled, unaware, or both? The better-than-average heuristic and statistical regression predict errors in estimates of own performance. Journal of personality and social psychology, 82 (2), 180-8 PMID: 11831408[g_100x25_D] [g_100x25_D] Burson KA, Larrick RP, & Klayman J (2006). Skilled or unskilled, but still unaware of it: how perceptions of difficulty drive miscalibration in relative comparisons. Journal of personality and social psychology, 90 (1), 60-77 PMID: 16448310[g_100x25_D] [g_100x25_D] Share this: * Tweet * * * * Share on Tumblr * Related Posted on July 7, 2010July 7, 2010Author Tal YarkoniCategories psychology, research, statisticsTags confirmation bias, decision making, dunning-kruger effect, metacognition, regression to the mean, self-enhancement 90 thoughts on "what the Dunning-Kruger effect is and isn't" Comment navigation Older Comments 1. Pingback: D-K Thoughts - Mershon's Musings 2. [27f2ad][27f2ad] Butul Miah says: May 2, 2017 at 11:23 am If the task is not in any way life-threatening, then it's better to throw caution to the wind. You will learn much more, far more quickly, with more robustness, if you are extremely bold. Hence, delusional self-confidence [i.e. Dunning-Kruger's "over-confident incompetents"] is the better strategy, in this type of scenario. You will not explore with a deathly fear of making "mistakes", the delusional self-confidence will keep you trying at the task, and you will not give up out of desperation, giving yourself a better [though not perfect] chance of completing and mastering the task. With exploration may come enhanced techniques, which will further cement your mastery of a task, and fuel a cycle of confidence-begets-greater-learning-and-improvement. You DO have to have motivation at the task at hand, however. If you lose interest, that could trigger a decline in skills. Reply 1. [32d734][32d734] Regina Burns says: November 18, 2017 at 9:36 pm So this makes sense to me. In recent weeks I felt inspired to learn how to create a woodcut. There is so much to learn, but I am enjoying every moment of this process. And by pushing through the difficult, early scary parts of learning these new skills, believing that I will master this art even as I observe my own my incompetence keeps me interested. At the end of each carving session I make a rubbing of my small progress and am challenged to a higher level. Reply 2. [Pl_LEn][Pl_LEn] kruasan (@kruasan1) says: January 7, 2020 at 1:33 pm This is a good point, delusional self-confidence can be a helpful thing. When I started writing music, I was, in a sense, delusional. I thought my first songs were so great, which obviously wasn't the case. But it's that overly confident feeling of excitement and amusement that gave me the motivation to continue. So thinking that you are much better than you actually are can prevent loss of interest in an activity. Reply 3. Pingback: Our story: 10. President Ignoramus | HappyPeopleTrue_nJust 4. Pingback: Why incompetent people often think they're actually the best - News World Club 5. Pingback: OVER/UNDER/RATED EGOS - God's HotSpot 6. Pingback: Why do remote meetings suck so much? - Chelsea Troy 7. Pingback: Studie: Folk med liten kunnskap om politikk tror de kan mye | 8. Pingback: BRUG21: Daniel Johnston, Phil Ochs, Dunning-Kruger effect, Verzetsmuseum en het Klokkenschip - Brug Podcast 9. Pingback: Week 7: Read This Post In A Funny Accent - Unicorn Dispatches 10. Pingback: Bo no ta mesun sabi ku bo ta pensa! - BOHIMIO 11. Pingback: Intellectual humility: the importance of knowing you might be wrong - Latest Online News Website 12. Pingback: Intellectual humility: the importance of knowing you might be wrong | 13. Pingback: Intellectual humility: the importance of knowing you might be wrong - 14. Pingback: Intellectual humility: the importance of knowing you might be wrong - Original World News 15. Pingback: An expert on human blind spots gives advice on how to think - Non Perele 16. Pingback: Blog: An expert on human blind spots gives advice on how to think - Nur die Kirschen 17. Pingback: An expert on human blind spots gives advice on how to think - Latest Online News Website 18. Pingback: The Dunning-Kruger effect, and how to fight it, explained by psychologist David Dunning | 19. Pingback: An expert on human blind spots gives advice on how to think - 20. Pingback: ZOTESIA PER TE VLERESUAR PAZOTESINE - Media e lire - blog kolektiv shqiptar 21. Pingback: dunning kruger syndrome - Idhey 22. Pingback: Explaining The Dunning-Kruger Effect - Manage By Walking Around 23. Pingback: An expert on human blind spots gives advice on how to think 24. Pingback: An expert on human blind spots gives advice on how to think - AVConnecting 25. Pingback: The Dunning-Kruger effect, and how to fight it, explained by psychologist David Dunning - The Latest World News 26. [378436][378436] Feona says: September 8, 2019 at 5:18 pm Not having enough knowledge to know you don't have knowledge/not being smart enough to know you're not smart or too dumb to know you're dumb. Seems to be the core issue with the far right wing Trump supporters. It explains why they can't decipher facts from fiction. Reply 27. Pingback: Does pair programming work? - 28. Pingback: "How to Tell Your Story Without a Self-Evaluation" 29. Pingback: NC Weekly 9: The real fountain of youth - Why not knowing and changing keep us young. - Negative Capability 30. Pingback: Clinical Pearls and Advice: An Update Part 1 - Modern Pain Care Main 31. Pingback: What the Dunning-Kruger effect is and isn't - Site Title 32. Pingback: ACCURATE SELF REFLECTION - GOD'S HOTSPOT 33. Pingback: random links (56) | adrift 34. Pingback: "How to Tell Your Story Without a Self-Evaluation" - Ami Axomiya 35. Pingback: DUNNING-KRUGER EFFECT: IGNORANCE AND OVERCONFIDENCE AFFECT INTUITIVE THINKING, NEW STUDY SAYS - Gianni Bellorio Comment navigation Older Comments Leave a Reply Cancel reply Post navigation Previous Previous post: this year, i backed new zealand to go all the way Next Next post: elsewhere on the net, vacation edition Proudly powered by WordPress