[HN Gopher] Revisiting Stereotype Threat
___________________________________________________________________
Revisiting Stereotype Threat
Author : systemstops
Score : 56 points
Date : 2024-12-19 18:08 UTC (4 hours ago)
(HTM) web link (www.speakandregret.michaelinzlicht.com)
(TXT) w3m dump (www.speakandregret.michaelinzlicht.com)
| peterldowns wrote:
| Unsurprising. Although this is the first time I can recall
| reading a psychologist accept culpability for the field's bad
| "science" over the last twenty years. Have any of the
| "anthropology of science" researchers published an explanation of
| that yet?
| aprilthird2021 wrote:
| Everyone says "unsurprising" after the replication fails. Idk,
| this was a really popular theory that lots of people believed.
| I doubt you were all so confident back in the heyday of this
| social science phenomenon.
|
| If tomorrow, they say "growth mindset" is also a non-replicable
| phenomenon, will HN be full of smug people saying "I knew it
| all along, lol!"
| renewiltord wrote:
| That's just a heterogeneous population with selection bias.
| Everyone is always convinced because when they're disproven
| they stay silent. These days prediction markets allow people
| to convert beliefs to money so it becomes not very convincing
| when everyone manages to be perfect at predicting things and
| is always complaining they aren't making enough money while
| simultaneously giving up the chance at doubling their bet on
| a sure shot.
|
| Combine that with people upgrading uncertainty to certainty
| post-hoc when debunking comes out and you have these
| entertaining things. Overall, I'm glad you called it out.
| Once I wished I had a profile for people's past guesses to
| see how actually good they are and now I have Manifold,
| Kalshi, and Polymarket.
| Hasu wrote:
| I'm not a research psychologist, but my understanding is that
| growth mindset is already non-replicating.
| bluGill wrote:
| Even if you did suspect something was wrong you were careful
| because science said otherwise. Calling this wrong in public
| would feel like saying a big rock falls faster than a small
| rock - something else that seems obvious but science has
| disproved. And so the only one loudly saying this was bunk
| where the right wing crackpots (the reasonable people on the
| right side were much quieter about it because they don't like
| to argue with science even though it went against their bias)
| - the right wing crackpots were on this only because the left
| wing embraced it as a science that confirmed their bias.
| Unfortunately in this cast the conspiracy was true and so the
| crackpots won (even though they have no understanding of the
| real reasons it is false and it doesn't seem to have been a
| conspiracy).
|
| Let this be a lesson: even if something seems like science
| and it confirms you bias - that doesn't mean it is true. You
| should look closer at things you embrace than things you
| reject lest you embrace a lie.
| mistermann wrote:
| I'm a rather reasonable "on the right" person and arguing
| about science is literally my hobby, there are few funner
| things in the world imho.
| jdietrich wrote:
| It might have been surprising at the time, but it's certainly
| unsurprising in light of the replication crisis. We now know
| that most findings don't replicate. Anyone who continues to
| be surprised by _the most likely outcome_ needs to update
| their priors.
|
| https://en.wikipedia.org/wiki/Why_Most_Published_Research_Fi.
| ..
| neuronexmachina wrote:
| From your link:
|
| >Biostatisticians Jager and Leek criticized the model as
| being based on justifiable but arbitrary assumptions rather
| than empirical data, and did an investigation of their own
| which calculated that the false positive rate in biomedical
| studies was estimated to be around 14%, not over 50% as
| Ioannidis asserted.
| saurik wrote:
| FWIW, to me it is "unsurprising" as I can't remember the last
| time a major psych study actually did replicate, and it seems
| frankly like the entire field is in shambles. Does that mean
| that I _knew_ before that this study in specific was false?
| No. But, that isn 't what anyone means when they say
| "unsurprising"! If you get to the end of a movie and all of
| the major characters end up alive, and someone else seemed
| _shocked_ by that fact, you might still explain to them that
| that 's "unsurprising", even if you yourself got to feel the
| thrill of uncertainty -- or even were concerned for a bit --
| while you were watching.
| mistermann wrote:
| Be careful to not mistake "theory doesn't replicate" with
| "theory is not true [in whole or in part]".
| trhway wrote:
| Would you expect the theory of non replicative theories to
| be replicative or not?
| whatshisface wrote:
| I'm skeptical of all of these phenomena, they are like
| personality tests and astrology.
| zahlman wrote:
| Link for the claim that the results don't reproduce, without
| Facebook tracking: https://osf.io/preprints/psyarxiv/qctkp
|
| > When Black students at Stanford University were told that a
| test was diagnostic of intellectual ability, they performed worse
| than their white counterparts. However, when this stereotype
| threat was ostensibly removed--by simply framing the test as a
| measure of problem-solving rather than intelligence--the
| performance gap Black and white students nearly vanished.
|
| Just reading this description motivates me to reject the study
| out of hand. It's not plausible that university-level students
| responded meaningfully differently to being told "this is a test
| of problem-solving skill" versus "this is a test of intelligence"
| because _it is commonly understood that problem-solving skill is
| a major component of intelligence_.
|
| >it also became the darling of the political left who now had an
| answer to prevailing views of group differences held by the
| political right. This is partly because shortly before stereotype
| threat took its turn in the spotlight, Charles Murray and Richard
| Herrnstein published The Bell Curve... the octogenarian Murray is
| still considered a pariah, shouted down and deplatformed from
| talks he tries to deliver at respectable colleges to this day.
|
| The characterization of Murray's views in the last several years
| has been grossly uncharitable and seems entirely disconnected
| from his actual arguments. It's strange that the book is 30 years
| old, but has seemed politically relevant for much less time than
| that.
| aprilthird2021 wrote:
| Eh, I mean the stereotype being "activated" is that Black
| people are less intelligent / have lower IQs, not that they
| lack problem-solving skills.
|
| Even the replication attempt had two scenarios:
|
| 1 where women were told the test was to establish performance
| levels on the test between men and women
|
| 1 where they were told the test was a test of problem-solving
| skill (or primed to disregard negative stereotypes before the
| test).
|
| So even the replication has the incorrect framing you worried
| about. I tend to believe the problem wasn't this, but the way
| the field was lax about sampling, methodology, etc. After all,
| there were many stereotype threat studies, not just this one,
| boasting similar results. And they didn't all use that framing.
| rat87 wrote:
| The characterization of Murray's views seem extremely accurate
| and the fact that his book is still brought up despite being
| disproved as racist trash decades ago is ridiculous
| runamuck wrote:
| Executive Summary of Article: "new data now reveal what many of
| us suspected for at least ten years: stereotype threat does not
| replicate, and it does not undermine academic performance in the
| ways we thought."
|
| The Stereotype Threat: "individuals who are part of a negatively
| stereotyped group can, in certain situations, experience anxiety
| about confirming those stereotypes, leading paradoxically to
| underperformance, thus confirming the disparaging stereotype."
| for example, if you remind a woman of the "Women are bad at math"
| stereotype, they will perform worse on a math test than if they
| are not reminded of that stereotype.
| verteu wrote:
| Also interesting: p28 (labeled p470) of
| https://journals.sagepub.com/doi/pdf/10.1177/251524591881022...
| ("Many Labs 2: Investigating Variation in Replicability Across
| Samples and Settings") shows the results of attempting to
| replicate 28 published psych results across many different
| samples.
|
| Unfortunately, "Stereotype Threat" is not one of the effects they
| attempted to replicate.
| underlipton wrote:
| Putting on my contrarian hat: the studies he mentions seem to be
| concerned with whether testing conditions affect stereotype
| threat effects. Logically, they can't prove or deny the
| phenomenon's existence, only whether testing conditions result in
| changes to test outcomes that could track with interventions to
| reduce stereotype threat. Much of what we know about how
| behavioral effects of identity comes with the understanding that
| the latter is something people carry with them, regardless of
| local or recent events. If there's a problem with stereotype
| threat as a concept, it's that it's positioned as a superficial
| effector that can be manipulated easily, rather than the surface
| level manifestation of complex interactions between self-
| identity, personal values, and cultural expectations. Based on
| the author's disclosure about his PhD thesis, he seems to be
| someone who capitalized on the former characterization, so of
| course he throws the baby out with the bathwater when it no
| longer works for those purposes. We might be looking at an ass-
| covering write-up.
|
| >Let's play "Find the Lebowski quotes game" again!
|
| So, yeah, I find this a deeply unserious blog post.
| bluGill wrote:
| He must throw the baby out with the bathwater - so far there is
| no evidence for anything here. The only alternative is to rerun
| the study, doing it correctly this time and then do a correct
| analysis. Until then you need to say "I don't know" when asked
| about this as that is all science allows. Maybe the rest of the
| baby they are throwing out will replicate and thus is correct -
| but nobody knows that and so we cannot say anything with
| confidence right now.
|
| I'm assuming you are not aware of studies not mentioned here
| that replicate - I'm not in this field and so I would not know
| where to look. I'm guessing that you also are not in this field
| and are looking for some way to allow your bias to become true
| despite these issues - but of course I might be wrong.
| underlipton wrote:
| Let me analogize. This is a blog post that claims that racism
| isn't real. As proof, the author explains that none of the
| studies that show a reduction in racism when you remind
| people that Michael Jordan and Beyonce exist can be
| replicated. Well, okay, sure. And the guy who earned a
| doctorate with the thesis, "Does Playing Jackson 5 Make
| People Less Racist?" might want to turn this into, "Racism
| does not exist," as a way to distance himself from his own
| terrible scholarship. (Also, did you find his Marvel
| Cinematic Universe Easter Eggs?)
|
| So, thus far, there is actually not enough evidence to throw
| the, "Racism exists," baby out with the, "Do these
| interventions affect racist belief?" bathwater. And since
| we're second-guessing biases, it's super weird that you're
| always in comment sections of (politically-charged) articles
| concerning fields you're not in.
| shermantanktop wrote:
| I think that contrarian hat may include a no-true-scotsman
| badge.
|
| If stereotype threat is real, we should be able to have a
| replicatable study result that confirms it, right? We're not
| just limited to logical inference, I hope.
| underlipton wrote:
| If the studies are concerned with determining that stereotype
| threat exists, sure.
| whimsicalism wrote:
| > Let's be honest: that last sentence was far too generous. Many
| of us engaged in practices that, in hindsight, were borderline
| dishonest. We abused experimenter degrees of freedom, engaged in
| questionable research practices, p-hacked, massaged our data--you
| pick the euphemism. In contrast, this new replication study
| followed the most up-to-date best practices in psychological
| science, eliminating room for flexibility in analysis or results
| interpretation.
|
| Exploiting researcher degrees of freedom remains unfortunately
| extremely common. There needs to be some sort of statistical
| vanguard in the ivory towers enforcing _real_ preregistration and
| good analysis practices. Strict epistemic discipline is necessary
| to do real science.
| bluGill wrote:
| The real problem is we have a lot of data. So it is easy to
| shove it all into a computer and then see what comes out. This
| is not a bad thing, but it isn't alone a result to publish it
| is instead justification to do a real study on whatever thing
| statistics turns up as a significant result to make sure it
| really is when you properly isolate for that data.
| trhway wrote:
| It is interesting to see how those very sensitive to the Party
| line and very skillful in political games people start to change
| course - once they felt the political wind change with Trump and
| Musk ascending.
|
| That particular guy benefitted from peddling the darling theories
| of the day (and naturally, given how Big Academia is, was part of
| the oppression machine of anybody who dared to question those
| darling theories) and now, once the opposite of those theories
| became in political vogue, he rides the wave of dismantling them
| rushing full speed ahead of the pack to declare his loyalty to
| the new regime. Just like such people did in the old days of
| USSR.
| armoredkitten wrote:
| This has been an ongoing situation since around 2010. The field
| has been undergoing a huge shift in terms of re-evaluating
| research practices and past findings. It genuinely has nothing
| to do with national politics.
|
| Also, the author is Canadian. So there's that, too.
| pessimizer wrote:
| Stereotype threat is just one of a million ways to say that the
| position of Black people in US society, and women's position in
| world society, is their own fault without assigning them any
| guilt. Yes, they're worse off than everyone, but it's actually
| _no one 's fault._ Its a bunch of mindset. Have you heard of the
| law of attraction? This is the law of repulsion.
|
| The fact is, all you have to do is allow women to own property
| and inherit money for a few generations, and you'll see
| differences inherited from their status in society ironed away.
| If women own the place, they dictate the decisions. With the
| descendants of US slaves, all you have to do is make a single
| proportional effort to make up for the difference in the
| inheritance that they got from the dozen generations that worked
| for free, and the inheritances that the people get who profited
| from that. That's it, all excuses gone. The reality is that black
| people could actually be inferior to whites, most of us are of
| about 20% descent from people so low on the human scale that they
| would rape a slave.
|
| Academic America: Instead of fixing the problem, which is
| expensive, make a living off of mystifying it, which is
| lucrative.
| systemstops wrote:
| Inherited wealth was actually pretty rate in the history of
| this country, and is not as common now as many people think.
| Generational wealth until recently did not exist for most
| people.
| tashi wrote:
| I don't think that's what people have been taking away from the
| idea of stereotype threat at all. I think it's been much more
| likely for people to think, "Oh, instead of unchangeable
| genetic traits, the difference in outcomes might someday be
| reduced through deliberate social change.
| taeric wrote:
| I'm curious what the impact for most people's mindsets will be
| here? I'm imagining that it relates to what many people consider
| "talent" with kids? I know that has been an odd trend in some
| states trying to get rid of "gifted" programs and such. I have
| largely remained hopeful that that was not nearly as prevalent as
| online represents.
| akoboldfrying wrote:
| >this study (still a preprint) was preregistered (meaning all
| methods and analyses were specified before the data were
| collected)
|
| This is the way forward -- preregistered studies. That, together
| with a promise from the publisher to publish the result
| _regardless of whether the effect is found to be significant._
|
| When you think about it, the incentives for publishing in science
| have been wrong all along. The future will be different: It will
| be full of null results, of ideas people had that didn't pan out.
| But we'll be able to trust those results.
| ckemere wrote:
| I also have a high degree of skepticism about most psychology
| research. I find it frustrating that the author of this piece
| raises the issue of the imbalance of gender in STEM jobs as a
| reason to doubt this particular study. There is nothing about
| this failure-to-replicate that should allow us to conclude that
| innate differences in math ability underly that imbalance.
|
| I'd love others to read the replication report and explain why I
| might be wrong?
| ckemere wrote:
| I mean come on - if there is a pervasive cultural insinuation
| of gender differences in ability that leads to women
| experiencing more test anxiety (I'm not saying there is, but
| conditioning on that hypothesis) why would we expect a 20
| minute PowerPoint to counteract 20 years of people's life
| experience!?!?
| armoredkitten wrote:
| As someone who went through grad studies in the field, and who
| has met Dr. Inzlicht in person before, I have to say I deeply
| appreciate his perspective. He has consistently been humble when
| facing the issues in the field, in ways that call even his own
| previous research (following the typical practices of the day)
| into question. The field as a whole has been undergoing a
| reckoning, but Mickey has been one of the people who has
| encouraged his fellow researchers not just to wag their fingers
| at others, but also to look inward and reflect on their own
| research practices. He has done so by showing humility and
| acknowledging where his research has fallen short, and that
| indicates to me a great deal of integrity.
|
| It is sad to see stereotype threat being one of those findings
| that seems less and less credible. I once worked as a research
| assistant on a project related to stereotype threat, and I recall
| the study going through several iterations because it all needed
| to be _just so_ -- we were testing stereotypes related to women
| and math, but the effect was expected to be strongest for women
| who were actually good at math, so it had to be a test that would
| be difficult enough to challenge them, but not so challenging
| that we would end up with a floor effect where no one succeeds.
| In hindsight, it 's so easy to see the rationale of "oh, well we
| didn't find an effect because the test wasn't hard enough, so
| let's throw it out and try again" being a tool for p-hacking,
| file drawer effects, etc. But at the time...it seemed completely
| normal. Because it was.
|
| I'm no longer in the field, but it is genuinely heartening that
| the field is heading toward more rigour, more attempts to correct
| the statistical and methodological mistakes, rather than digging
| in one's heels and prioritizing theory over evidence. But it's a
| long road, especially when trying to go back and validate past
| findings in the literature.
___________________________________________________________________
(page generated 2024-12-19 23:01 UTC)