[HN Gopher] The Dunning-Kruger effect is autocorrelation
___________________________________________________________________
The Dunning-Kruger effect is autocorrelation
Author : ljosifov
Score : 179 points
Date : 2023-11-25 18:14 UTC (4 hours ago)
(HTM) web link (economicsfromthetopdown.com)
(TXT) w3m dump (economicsfromthetopdown.com)
| Jensson wrote:
| Psychologists using their pet theories to explain results and
| then people taking that explanation as the truth when they should
| really just look at the data is probably an as large problem as
| the replication crisis.
| glitchc wrote:
| Geez, this is eye-opening. Thank you for sharing this.
| tempestn wrote:
| I don't buy this take, and this rebuttal does a better job than I
| could of explaining why: https://andersource.dev/2022/04/19/dk-
| autocorrelation.html
|
| Basically, this autocorrelation take shows that if performance
| and evaluation of performance were random and independent, you
| would get a graph like the D-K one, and therefore it states that
| the effect is just autocorrelation. But in reality, it would be
| very surprising if performance and evaluation of performance were
| independent. We expect people to be able to accurately rate their
| own ability. And D-K did indeed show a correlation between the
| two, just not as strong of one as we would expect. Rather, they
| showed a consistent bias. That's the interesting result. They
| then posit reasons for this. One could certainly debate those
| reasons. But to say the whole effect is just a statistical
| artifact because random, independent variables would act in a
| similar way ignores the fact that these variables aren't expected
| to be independent.
| Jensson wrote:
| The effect that the worst overestimate their skill is known
| since before, that wasn't the main result of Dunning-Kruger.
| The effect that the best underestimate their skill can be
| chalked up to auto-correlation.
| tempestn wrote:
| The best don't tend to overestimate their skill; they
| underestimate it. The D-K results show a consistent bias in
| estimates toward (somewhere near) the mean. Hence an
| overestimate at the bottom and an underestimate at the top.
| Jensson wrote:
| > The best don't tend to overestimate their skill; they
| underestimate
|
| I wrote the wrong word, I fixed it. The best can't
| overestimate their rank, so of course that wasn't what I
| meant.
| anonymouskimmer wrote:
| Dunning-Kruger posits this as a psychological effect, yes?
| On the top half psychological effects such as imposter
| syndrome could come in to play.
|
| Have sociological factors such as being kind or big fish
| little pond been considered as likely causes of the
| misestimates?
| svnt wrote:
| The author of this assumes the conclusion in order to decide
| how to analyze his data.
|
| He cannot reasonably say both:
|
| > we have a decision to make: what are we going to assume? How
| are we going to quantify our surprise from the results?
|
| > The first option is, as in the case of the state census, to
| assume dependence between X and Y. I.e. to assume that,
| generally, people are capable of self-assessing their
| performance.
|
| > The second option conforms with the Research Methods 101
| rule-of-thumb "always assume independence." Until proven
| otherwise, we should assume people have no ability to self-
| assess their performance.
|
| > It seems to me glaringly obvious that the first option is
| much, much more reasonable than the second.
|
| -- and -
|
| > most notably the claim that the more skilled people are, the
| better they are at self-assessing their performance. This
| result is supported by their plot, but in any case, my issue is
| not with objections to this claim
|
| and then expect to carry any credibility.
|
| The author of this piece both suggests that a key variable is
| fixed and later admits it varies within the same dataset.
|
| I guess at least they admit it, but this lacks basic self-
| consistency.
| Jensson wrote:
| > The author of this piece both suggests that a key variable
| is fixed and later admits it varies within the same dataset.
|
| I don't see how that variable changes, here is an example how
| the error variable can be exactly the same for everyone and
| reproduce the results:
|
| Lets say the overconfidence is always that you feel 50% of
| those better than you are actually worse than you. So
| everyone is equally overconfident, just that the top wont
| move their own placings as much as the bottom since there are
| much fewer people that they can mistake being worse than
| them. Then apply noise to this and you get the graph Dunning-
| Kruger got.
|
| You could say "But they are better at estimating their
| rank!", but that is just a mathematical artefact, it isn't a
| psychological result. Even if everyone always guessed that
| they are number 1, the better you are the better your guess
| will be, but in that case it is easy to see that everyone
| overestimates their skill in the same way instead of the
| better people having a fundamentally different way of
| evaluating themselves.
| atleastoptimal wrote:
| The issue is people have differing personal definitions of
| Dunning Kruger. The generally demonstrated effect in the sample
| of people Dunning and Kruger analyzed was "people tend to
| estimate the percentile of their own skill as closer to the
| average than it really is, with a slight bias towards an above-
| average mean. This leads to overestimation of relative ability
| by those in lower percentiles, and the opposite for those in
| higher percentiles"
|
| However when people cite Dunning Kruger in popular culture they
| mean "below average people think they're above average, and
| above average people assume they're below average", which was
| not shown in the original study, and wouldn't show up in an
| analysis attempting to justify it via a misunderstanding of
| autocorrelation.
|
| The general point in the rebuttal is correct. A completely
| noisy graph of people's estimations of their own ability would
| show a Dunning-Kruger resembling residual graph (x-y vs x).
| However, one wouldn't expect people in the 1st percentile to
| have an equal distribution of perceived skill as people in the
| 50th or 99th percentile. If that were true, it would be worth
| reporting.
| ShamelessC wrote:
| > "below average people think they're above average, and
| above average people assume they're below average"
|
| There's no way to know if you're wrong, but when I see it
| used it seems to be pointing out - "some (not all) under
| qualified people tend to defer to their own beliefs rather
| than the views/statements from experts, even when that is
| demonstrably silly."
| staunton wrote:
| Which also has nothing at all to do with this study by
| Dunning and Kruger. So you agree with the general point of
| parent.
| ShamelessC wrote:
| Yes. Just clarifying a small disagreement about the pop-
| sci interpretation of the phrase.
| crazygringo wrote:
| Yup. Assuming the sample sizes are statistically significant,
| the original paper clearly shows:
|
| - On average, people estimate their ability around the 65th
| percentile (actual results) rather than the 50th (simulated
| random results) -- a significant difference
|
| - That people's self-estimation _increases with their actual
| ability_ , but only by a surprisingly small degree (actual
| results show a slight upwards trend, simulated random results
| are flat) -- another significant difference
|
| The author's entire discussion of "autocorrelation" is a red
| herring that has nothing to do with anything. Their randomly-
| generated results do _not_ match what the original paper shows.
|
| None of this really sheds much light on to what degree the
| results can be or have been robustly replicated, of course. But
| there's nothing inherently problematic whatsoever about the way
| it's visualized. (It would be nice to see bars for variance,
| though.)
| IAmGraydon wrote:
| So what we have here is some scientists trying to prove that
| the Dunning-Kruger effect doesn't exist and instead they give
| us a perfect example of the Dunning-Kruger effect.
| wyldfire wrote:
| > The irony is that the situation is actually reversed. In
| their seminal paper, Dunning and Kruger are the ones
| broadcasting their (statistical) incompetence by conflating
| autocorrelation for a psychological effect. In this light,
| the paper's title may still be appropriate. It's just that it
| was the authors (not the test subjects) who were 'unskilled
| and unaware of it'.
| t_mann wrote:
| I was surprised by the figure from the original article, imho
| that's the strongest rebuttal: perceived ability grows strictly
| mononotonically with actual ability, no sign of the famous non-
| monotonic U-curve. Yeah, the slope is less than one, and it
| grows a bit faster from the second to the third quartile than
| from the first to the second, but none of that changes the fact
| that people tend to slot themselves correctly. The chart is
| interesting in that it confirms that everyone perceives
| themselves to be slightly above average in terms of ability,
| which of course can't be true in practice. But what it also
| shows is that when they think they'll be below or above that
| (false) baseline, they're actually correct about it. So pretty
| much the exact opposite of what the Dunning-Kruger effect
| claims.
| dmbche wrote:
| Isn't it ironic that they fooled themselves?
| ulizzle wrote:
| It was actually hilarious but I don't think many people here
| got the irony
| lencastre wrote:
| Wasn't this DK effect already debunked?
| jahewson wrote:
| I don't know much about it but I'm sure you're right.
| hasch wrote:
| Article mentions 2016 somewhere. They explain a bit on top of
| that, with more depth ... at least my rough take on this
| xbar wrote:
| Yes. This article highlights the 2016, 2017 and 2020 debunkings
| of DK. But it hangs on as an oft repeated scientific fallacy.
|
| The fact that anyone has to ask if it has debunked shows how
| desirable some people find the DK myth. Even in the comments
| here, people are not willing to be skeptical of DK. That's
| interesting psychology.
| mrkeen wrote:
| Yes but some claim to have debunked the debunking also. [1]
|
| This paper (2023) claims "the magnitude of the effect was
| minimal; bringing its meaningfulness into question." [2]
|
| [1] https://andersource.dev/2022/04/19/dk-autocorrelation.html
|
| [2]
| https://www.sciencedirect.com/science/article/abs/pii/S01602...
| pie_flavor wrote:
| This take is a perfect example of Dunning-Kruger itself,
| ironically. https://andersource.dev/2022/04/19/dk-
| autocorrelation.html
| dahart wrote:
| How so? DK shows a positive correlation between confidence and
| competence.
| mewpmewp2 wrote:
| My take on Dunning Kruger:
|
| 1. People really like the idea of smart people being humble and
| arrogance meaning stupidity, so they like to believe that DK is
| true, and they like to repeat this.
|
| 2. Some smart/skilled people are humble, some are arrogant.
|
| 3. Some smart/skilled people underestimate their skills, some
| overestimate.
|
| 4. Some stupid people are humble, some are arrogant.
|
| 5. Some stupid people underestimate their skills, some
| overestimate.
|
| Overall, even if there is a correlation, you can't tell by just
| arrogance of a person whether we are dealing with DK or whether
| it's an effect at all. People's personalities, skills and
| everything are a bit more complex than that.
|
| Overall bringing DK up seems like some sort of social
| justice/fairness effort rather than something that is actually
| true given any situation where someone is arrogant.
| spacebacon wrote:
| Maybe this shows how effective dumb people are at keeping smart
| people hammered down with thought stopping arguments.
| greenthrow wrote:
| Lmao this article is an example of Dunning-Kruger at work. The
| author thinks they have found and are revealing something but
| they are just failing to fully understand the subject. Amazing.
| flappyeagle wrote:
| Try reading the article again and understanding the argument.
| greenthrow wrote:
| Oh I did. Completely.
| joefourier wrote:
| So from my understanding, the Dunning-Kruger Effect paper doesn't
| show the distribution of the perceived test scores nor the
| standard deviation, only an average, which rises with actual test
| score level.
|
| If they showed the spread bar in each bin, you could form very
| different conclusions. Do low skilled people consistently
| estimate their score at around 60, or do they give effectively
| random results centred around 60?
|
| Assuming the latter, it could mean that low skilled individuals
| are completely unable to evaluate their performance while higher
| skilled people are slightly better at it but still not very good,
| giving a slightly positive correlation which... is very distinct
| from what the DK effect implied.
| xanderlewis wrote:
| Naive take: I've always felt like Dunning-Kruger is just the
| result of the fact that when guessing the value of anything
| people tend towards some common mean, and so if the true value is
| low your guess tends to be high, and vice versa. This assumes
| nothing about what is being guessed, but does assume (perhaps
| wrongly) that there is a commonly believed mean value and that
| people tend to imagine they are close to it.
| wavemode wrote:
| That's essentially the plain-language interpretation of what
| the author of this article is pointing out - when you plot
| (actual score) against (difference between test score and
| actual score), you will always find a trend that
| underperformers overestimate and overperformers underestimate -
| for the exact reason you state.
| r0uv3n wrote:
| The discussion between Nicolas Boneel and the author in the
| comments of the article is interesting and Nicolas expresses the
| doubts I had when reading this. The whole point of the DK effect
| is that people are bad at estimating their skill, so if you
| assume that they randomly guess their skill level then of course
| you will replicate the results.
|
| The correct model for a world without DK should be something like
| (estimated test scores)=(actual test scores)+noise, and then the
| only form of spurious DK you'd expect is caused by the fact that
| there's a minimum and maximum test score. But this effect would
| be proportional to the variance of the noise, and I assume the
| variance on the additional dataset is too low to fully understand
| the effect seen there.
|
| Also, in this model on average everyone should still guess
| correctly in which half of the distribution they are, but even
| the bottom quartile seemed to estimate their abilities as above
| the 50th percentile
| svnt wrote:
| Just because the data appear random doesn't mean you've gotten
| at the cause though.
|
| From those charts it could equally be low skill throughout, or
| something nuanced like lack of skill at estimating at the
| bottom, improving skill in estimating through the middle, and
| high skill and learned modesty at the top.
| Jensson wrote:
| > Also, in this model on average everyone should still guess
| correctly in which half of the distribution they are, but even
| the bottom quartile seemed to estimate their abilities as above
| the 50th percentile
|
| Depends on the noise applied. If the noise is -10% to +100% for
| everyone then you get roughly the graph Dunning-Kruger got. So
| there is no reason to believe that the best are better at
| estimating their abilities, just that you can't estimate your
| own rank as better than the best.
| tempestn wrote:
| That's a great observation. For what it's worth though, it
| does seem logical to me that the best would also be best at
| estimating their skill. Not necessarily because they're
| better at it per se (though there's likely some of that too,
| for the reasons originally posited by D-K), but also because
| they have an easier problem to solve. When you know something
| well, it's fairly obvious that that's the case. (Think of the
| experience of acing a math test. It's entirely possible you'd
| know you answered everything correctly.) When you struggle
| somewhat though, it's much more difficult to estimate how
| much you're struggling compared to how others would fare.
| hn_throwaway_99 wrote:
| Previous discussion:
| https://news.ycombinator.com/item?id=31036800
| bitshiftfaced wrote:
| The authors did "X - Y vs X," but that's not even the biggest
| problem. The authors subtracted two measures that had been
| transformed and bounded from 0 to 1 (think percentiles). What
| happens at the extremes of those bounds? How much can your top
| performers overestimate their performance? They're almost at 1
| already, so not much. If they were to overestimate and
| underestimate at the same rate and by the same magnitude in terms
| of raw values, the ceiling effect on the transformed values means
| that the graph will make it look like they underestimate more
| often. The opposite problem happens for the worst performers.
|
| See "Random Number Simulations Reveal How Random Noise Affects
| the Measurements and Graphical Portrayals of Self-Assessed
| Competency." Numeracy 9, Iss. 1 (2016), particularly figures 7,
| 8, and 9.
| anonymouskimmer wrote:
| This can be dealt with to an extent by truncating the extreme
| ends. Even the middle quartiles in the graphs in the linked
| article show the same trends.
| ImaCake wrote:
| Thanks for stating just how much of a statistical minefield
| this is. The reference does a great job showing just how wrong
| the DK studies are. Unfortunately, most people have already
| made up their minds and are happy to link conflicting blog
| posts as evidence.
| Probiotic6081 wrote:
| Probably in another year or two they'll find another
| statistic that will render the old one moot like again and
| again.
| dclowd9901 wrote:
| I think if people at all levels of skill were reasonably good
| at measuring their own ability, we would see two curves that
| roughly overlap. Instead we see the graph given.
|
| The fact that random noise can generate a mean curve on the Y
| axis doesn't mean DK doesn't exist. It just means DK's mean
| self analysis resembles a middling random mean, which if you
| think about it, makes sense. Most people will probably self
| evaluate as average, regardless of their actual skill. This
| means DK is right as rain.
| SamBam wrote:
| Exactly, that was my thought. How would it be _possible_ to get
| anything other than the D-K effect, even if it wasn 't just
| averaging to the mean?
|
| The lowest quartile can't say they're below the lowest
| quartile, so any error at all will be counted as
| "overconfidence." The top quartile can't say they're above the
| top quartile, so any error at all will be counted as
| "underconfidance."
| chiefalchemist wrote:
| DK for me is simply: "You don't know what you don't know." When
| that happens, it's easy - surprise, surprise! - to misjudge your
| skill level. In a way, it almost feels cruel to ask someone with
| too few points of reference to say how much they know. The fact
| is whether high, low, or in the middle...they are guessing.
|
| On the other hand, with enough experience the depth and breadth
| of your context improves, as it should. At that point, mis-self-
| assessment is the result of arrogance, bravado, etc. That's a
| different problem than simply not knowing.
|
| If nothing else, DK has a case of apple v oranges.
| thewanderer1983 wrote:
| The Dunning-Kruger effect isn't as the article first quotes. It's
| an effect that everyone experiences. We as humans tend to over
| simplify things we don't understand well or at all. Therefore we
| over estimate our expertise on these subjects. We also tend to
| under estimate how much an expert on subjects we do know well.
| Everyone does this. It's not just dumb people.
| Jensson wrote:
| > We also tend to under estimate how much an expert on subjects
| we do know well
|
| Any evidence for this, except Dunning-Kruger? To me it looks
| like everyone overestimates themselves. There are a lot of
| professionals who think they are undervalued and that people
| worse than them gets all the rewards and fame.
| vismwasm wrote:
| The author measures the Dunning Kruger effect on his random data
| exactly because he assumes it when generating his random data.
|
| By modelling skill and perceived skill as uniform draws between 0
| and 100, the unskilled (e.g. skill=0) will over-estimate their
| skills (estimated skill = 50, the mean on the uniform random
| variable) and the skilled (e.g. skill=100) will underestimate it
| (as 50 as well, again the mean of the same random variable). The
| only ones who will be correct (on average) are the average
| skilled ones (skill=50).
| beltsazar wrote:
| I don't know if I agree that it's an autocorrelation, but one way
| to explain The Dunning-Krugger Effect is by acknowledging this
| simple fact:
|
| Most people think that they are an average person, but they can't
| be all average--there must be some people substantially below the
| median. Therefore, those people must overestimate their
| abilities.
|
| This also applies to other aspects, such as attractiveness. Less
| attractive people would overestimate their attractiveness.
| anonymouskimmer wrote:
| For all of the tests and rebuttals of the Dunning-Kruger effect
| the people tested are not drawing from the totality of other
| people, but trying to compare themselves solely to those who
| also took the same test.
|
| Anyone in a position to take such a test is almost guaranteed
| to be above average compared to the general population (which
| includes babies for intellectual tests, or the extremely old
| for attractiveness tests).
|
| I think this complicates personal evaluation.
| salty_biscuits wrote:
| It's just correlation, why do they keep calling it
| autocorrelation.
| snarkconjecture wrote:
| Nonstandard terminology warning: the author is using
| "autocorrelation" in a way I've never seen before. There is a
| much more common usage of "autocorrelation" to refer to the
| correlation of a timeseries with itself (shifted by some amount).
|
| If you use autocorrelation to refer to the thing in OP, you'll
| probably confuse people who know statistics, and vice versa.
| anonymouskimmer wrote:
| > If the Dunning-Kruger effect were present, it would show up in
| Figure 11 as a downward trend in the data (similar to the trend
| in Figure 7). Such a trend would indicate that unskilled people
| overestimate their ability, and that this overestimate decreases
| with skill. Looking at Figure 11, there is no hint of a trend.
|
| There certainly _is_ a hint of a trend. Why do people, when
| visualizing data with a distinct trend, say that because the
| "error bars" from a particular statistical test overlap zero that
| no trend exists!?
|
| Freshman _trend_ to over-confidence. Grad students _trend_ to
| under-confidence. Undergrads in general _trend_ to over-
| confidence (though this trend decreases as year in school
| increases), and post-graduates, whether grad students or
| professors, trend to under-confidence.
|
| These "trends" are not statistically significant, but they
| certainly are a trend!
|
| Also, the random data distribution in figure 9 doesn't show the
| same trends as Dunning-Kruger's curve in figure 2. Perhaps there
| is at least one psycho-social mechanism here worth investigating?
| mrkeen wrote:
| > These "trends" are not statistically significant, but they
| certainly are a trend!
|
| This is an oxymoron.
| Dylan16807 wrote:
| Oxymorons only sound contradictory on a surface level.
|
| Something "certainly" being a "trend" is the definition of
| statistical significance, so this is a straight up
| contradiction.
| anonymouskimmer wrote:
| See here: https://news.ycombinator.com/item?id=38416858
|
| "Trend" has multiple meanings. Statistics doesn't get to
| claim all of the meaning.
| anonymouskimmer wrote:
| Show how.
|
| I place mechanistic theory prior to statistics in science.
| Mechanistic theory can be tested, statistics are a kind of
| test.
|
| If a statistically-insignificant result shows consistent,
| though non-significant deviations, such as the kind seen in
| Figure 11, then it tells me it's worth investigating whether
| mechanism(s) are explaining a very small portion of the
| variation that will not, in itself, show up as statistically
| significant, as it's being swamped by variation in other
| parameters.
| Dylan16807 wrote:
| Consistency is a synonym for statistical significance. If
| there's consistency beyond random alignment, then there
| should be a statistical test you can apply over your data
| to extract the signal.
|
| You can extract surprisingly small signals relative to
| variation in other parameters. But if it's _actually_
| swamped, then it might not be real, so go get more data.
| anonymouskimmer wrote:
| > Consistency is a synonym for statistical significance.
|
| So basically you're telling me that if I can visually see
| a consistency that does not show up in their statistical
| test, then they aren't running an appropriate statistical
| test on what I'm seeing.
|
| > But if it's actually swamped, then it might not be
| real, so go get more data.
|
| Even better to design other experiments.
| Dylan16807 wrote:
| > So basically you're telling me that if I can visually
| see a consistency that does not show up in their
| statistical test, then they aren't running an appropriate
| statistical test on what I'm seeing.
|
| _Either_ they 're not doing the right statistics, _or_
| it 's a "consistency" that is much more likely to show up
| randomly than you naively expect, and the study needs to
| be repeated or enhanced.
|
| Sometimes you can see a pattern that's just a figment of
| chance. See also: numerology, jelly bean xkcd
| Dylan16807 wrote:
| If they're actually error bars, you can shrink them with more
| data. That will turn the hint of a trend into an observation of
| a trend. If it wasn't random noise giving a fake hint.
| anonymouskimmer wrote:
| > If they're actually error bars, you can shrink them with
| more data.
|
| Assuming the new data has the same systemic or instrumental
| bias as the old data. Even using a different test date could
| skew results enough to widen the error bars.
| abnry wrote:
| If there is a linear relationship between test score (X, ability)
| and test score self-assessment (Y, self-perception), then the
| random variables are modeled as:
|
| $$ Y \sim aX+b+N $$
|
| Where N is some statistically independent noise, mean zero.
|
| This means the covariance between them is
|
| $$ Cov(Y-X,X) = E[ ((a-1)X+b+N -(a-1)E[X]-b) (X - E[X]) ] $$
|
| Which is
|
| $$ Cov(Y-X,X) = E[(a-1)(X-E[X])(X-E[X])] + E[N(X-E[X])]= (a-1)
| Var[X] $$
|
| To get a "DK effect" we need (a-1) < 0, or a < 1. If a=0, in the
| case of the blog post, then this is absolutely true. If a=1
| (which, along with b=0, is the ideal scenario), then this is
| barely not true. If a > 1, then we'd have a whole new effect
| about arrogant experts.
|
| So the only thing that matters from this "auto-correlation
| perspective" is the rate at which an individual's self-assessment
| increases with their ability. As long as they underestimate the
| increase, a "DK effect" will occur.
|
| However, in the above analysis, we ignored the variable b. If a =
| 0.8 and b=0, we'd never have the so-called "DK effect" even
| though it matches the "auto-correlation perspective" because
| everyone would underestimate their ability.
|
| This tells me that the value of b matters. It is sort of like the
| prior ability everyone assumes they have. What the DK papers
| shows is that b > .5, which I think is in line with the spirit of
| the popular interpretation of the "DK effect". People should not
| be assuming they have, at a minimum, a capacity higher than the
| average.
|
| At the same time, the value b isn't insanely higher than .5,
| which also makes me want to cut those unskilled and unaware some
| slack. It "seems reasonable" to assume your baseline is average.
| That can't be the case, but it feels intuitive.
| concordDance wrote:
| The author fails to make his point quite badly. Of course if
| everyone's self assessment was random the bottom quartile would
| overrate themselves! And that would be half of the Dunning-Kruger
| effect and we could truthfully say "the bottom quartile of people
| overrate themselves"!
|
| The other part where those at the top have a better idea or where
| they rank noticeably does not come out in his toy example.
|
| Honestly, he comes across as not having the slightest
| understanding of how people interpet those graphs...
| im3w1l wrote:
| It's fascinating how great Elo and similar ranking systems are at
| curbing DK. You just get a number, and that's how good (bad) you
| are. It's incredibly precise too, there's just no arguing with
| it.
|
| Also since the topic is D-K I'm a bit scared that I'm the fool
| here, but isn't he misusing the term autocorrelation? What he
| describes sounds like just normal correlation?
| toasted-subs wrote:
| Idk I genuinely feel like after having to deal with 10+ doctors
| who all had different opinions. The last doctor finally made the
| same conclusion as me and he was the last person I had to see.
|
| There's always exceptions. And sometimes reading publications
| pertaining to a very specific thing should give you more say on a
| subject.
|
| I just feel bad American tax payer money and the best years of my
| life was spent on telling medical professionals they don't know
| what they are talking about.
| dclowd9901 wrote:
| I think what this article is missing is "the chart DK should have
| used."
|
| Instead we get a spurious explanation that doesn't make a lot of
| sense based on completely fabricated data. It's entirely natural
| for something that looks like DK to emerge from randomized data,
| especially when the Y axis is represented by some number of the
| mean (actually 50ish in this case).
| a-dub wrote:
| i think of acf as a measure of repeating temporal structure and
| how "strong" and "long" it is, if it exists.
|
| that is, it gives you a notion of if and what order of an ar
| model should fit any repeating structure in the data.
| randomizedalgs wrote:
| Consider the imaginary world that the author describes, in which
| people's estimate of their score is independent of their actual
| score. Wouldn't it be fair to say that, in this imaginary world,
| the DK effect is real?
|
| The point of the effect is that people who score low tend to
| overestimate their score and people who score high tend to
| underestimate. Of course there are lots of rational reasons why
| this could occur (including the toy example the author gave,
| where nobody has any good sense of what their score will be), but
| the phenomenon appears to me to be correct.
| ezekiel68 wrote:
| > However, there is a delightful irony to the circumstances of
| their blunder.
|
| Indeed. And I find the tendency of people in this comment section
| to defend the flawed theory is further confirmation of another
| scientific finding: that we decide based on emotion and then
| justify our decision using rationality.
| notShabu wrote:
| every domain of expertise has two "elo" systems, the niche one
| and the broader one.
|
| e.g. you can learn basic juggling in 30 minutes that you are top
| 10% of your friends/colleagues etc...
|
| however within the juggling community itself this is known as the
| "3 ball cascade" a really simple trick relative to the ones that
| requires years to master. an outsider may not be able to tell the
| difference between the 1 year expert and the 10 year master.
|
| a lot dunning-kruger can be explained by people in one or the
| other not understanding the other system
| lopatin wrote:
| Oh I read about the about the DK effect a while ago. I'm pretty
| much an expert in Psychology now, AMA.
| eagerpace wrote:
| Is this the opposite of imposter syndrome?
| markhahn wrote:
| the numeric experiment does not produce a line identical to what
| DK report. if DK's line where horizontal at 50%, it would indeed
| be nothing but autocorrelation.
| dahart wrote:
| Most people, even here on HN, do not know what the DK effect
| actually claimed to show. It does not show that confident people
| are more likely to be incompetent. Their primary result shows a
| positive correlation between confidence and supposed skill. (What
| skill, you ask?*)
|
| This article suggests DK is even simpler than autocorrelation,
| that it's just regression toward the mean.
| https://www.talyarkoni.org/blog/2010/07/07/what-the-dunning-...
|
| I don't know which statistical artifact it is, but I am quite
| convinced that the so-called DK effect is not demonstrating
| something interesting about human psychology, I don't buy that
| this is a real cognitive bias. I've read the paper several times,
| and the methodology seems to be lacking rigor. They tested a
| small handful of Cornell undergrads volunteering for extra
| credit, not a large sample, not the general population, and
| tested _nobody_ who actually fits the description of
| 'incompetent' in a meaningful way. They primarily measured how
| people rank each other, not what their absolute skill was - and
| ranking each other requires speculating on the skills of others.
| There are obvious bias problems with asking a group of pampered
| Ivy League kids how well they think they rank.
|
| * One of the four "skills" they measured was ability to get a
| joke - "appreciation of humor" - Huh? This is subjective! The
| jokes used aren't given in the paper, either. Another was
| 'grammar' tests.
| TrackerFF wrote:
| The DK effect has gotten WAY more cred than it should. Today, it
| is just anoter feel-good piece that people use to justify their
| feeling that they're (ironically) surrounded by loud idiots.
| austin-cheney wrote:
| The best way to differentiate DK from autocorrection is motive.
| Low performance people will focus on motives that reinforce the
| perception of their competence, for example preferring code style
| over code delivery because while both may be arguably important
| one requires less effort and risk to attain.
|
| There is research to qualify this out of Stanford. People will
| shift motives to attain complements and the types of compliments
| received will dictate the challenges they are willing to accept.
| When a compliment is specific to an action and measurable people
| will strive for continuously more challenging tasks to
| continually receive specific compliments. When compliments are
| generic and directed to the person they will tend to preference
| progressively less challenging tasks so that they continue to
| shine relative to the attempted effort. The differences in
| behavior produces a natural Dunning-Kruger effect wherein people
| seeking less qualified activities are more likely to over
| estimate their potential and degree of success.
|
| This also statistically verified in research that correlates
| predictions to confidence. The more confidence a person is in
| their predictions, such as political talk radio hosts, the less
| accurate their predictions tend to be.
| James_K wrote:
| I think the issue here is a confusion about what "bias" means. If
| they are self-assessing at random, then the high performers will
| all underestimate themselves, but this is not a bias towards
| underestimation as they are choosing randomly.
|
| That said, the chart from D-K seems to show a different bias and
| line up roughly with what you would expect. Someone with no
| knowledge assumes they are average skill and hence inflates their
| position, someone who is very good doesn't want to rate
| themselves the best because they assume others know as much as
| they do. The assumption underlying both groups is that you are
| normal and others are similar to you.
|
| I hypothesise that most people think they're average, which is
| something you could easily test by asking them to rate how well
| they think the average person would do on a test and comparing it
| to that individual's test score. I'm almost certain that high
| performers will overestimate the average, and low performers
| underestimate it.
___________________________________________________________________
(page generated 2023-11-25 23:00 UTC)