[HN Gopher] Revisiting Stereotype Threat
___________________________________________________________________
Revisiting Stereotype Threat
Author : systemstops
Score : 91 points
Date : 2024-12-19 18:08 UTC (1 days ago)
(HTM) web link (www.speakandregret.michaelinzlicht.com)
(TXT) w3m dump (www.speakandregret.michaelinzlicht.com)
| peterldowns wrote:
| Unsurprising. Although this is the first time I can recall
| reading a psychologist accept culpability for the field's bad
| "science" over the last twenty years. Have any of the
| "anthropology of science" researchers published an explanation of
| that yet?
| aprilthird2021 wrote:
| Everyone says "unsurprising" after the replication fails. Idk,
| this was a really popular theory that lots of people believed.
| I doubt you were all so confident back in the heyday of this
| social science phenomenon.
|
| If tomorrow, they say "growth mindset" is also a non-replicable
| phenomenon, will HN be full of smug people saying "I knew it
| all along, lol!"
| renewiltord wrote:
| That's just a heterogeneous population with selection bias.
| Everyone is always convinced because when they're disproven
| they stay silent. These days prediction markets allow people
| to convert beliefs to money so it becomes not very convincing
| when everyone manages to be perfect at predicting things and
| is always complaining they aren't making enough money while
| simultaneously giving up the chance at doubling their bet on
| a sure shot.
|
| Combine that with people upgrading uncertainty to certainty
| post-hoc when debunking comes out and you have these
| entertaining things. Overall, I'm glad you called it out.
| Once I wished I had a profile for people's past guesses to
| see how actually good they are and now I have Manifold,
| Kalshi, and Polymarket.
| Hasu wrote:
| I'm not a research psychologist, but my understanding is that
| growth mindset is already non-replicating.
| bluGill wrote:
| Even if you did suspect something was wrong you were careful
| because science said otherwise. Calling this wrong in public
| would feel like saying a big rock falls faster than a small
| rock - something else that seems obvious but science has
| disproved. And so the only one loudly saying this was bunk
| where the right wing crackpots (the reasonable people on the
| right side were much quieter about it because they don't like
| to argue with science even though it went against their bias)
| - the right wing crackpots were on this only because the left
| wing embraced it as a science that confirmed their bias.
| Unfortunately in this cast the conspiracy was true and so the
| crackpots won (even though they have no understanding of the
| real reasons it is false and it doesn't seem to have been a
| conspiracy).
|
| Let this be a lesson: even if something seems like science
| and it confirms you bias - that doesn't mean it is true. You
| should look closer at things you embrace than things you
| reject lest you embrace a lie.
| mistermann wrote:
| I'm a rather reasonable "on the right" person and arguing
| about science is literally my hobby, there are few funner
| things in the world imho.
| philwelch wrote:
| > Even if you did suspect something was wrong you were
| careful because science said otherwise.... And so the only
| one loudly saying this was bunk where the right wing
| crackpots (the reasonable people on the right side were
| much quieter about it because they don't like to argue with
| science even though it went against their bias) - the right
| wing crackpots were on this only because the left wing
| embraced it as a science that confirmed their bias.
|
| If "crackpots" turn out to be right when "reasonable
| people" and "science" were wrong--and this is far from the
| only instance of this happening--maybe we should reevaluate
| some things.
|
| Just because a peer reviewed paper published in a
| prestigious journal says something doesn't mean it's true.
| Even a survey of multiple peer reviewed papers published
| over time isn't necessarily determinative if there are
| common methodological issues or publishing biases. Yes, a
| lot of times actual crackpots will make stupid criticisms,
| but not every criticism is stupid even if it comes from
| outside the ivory tower.
|
| In particular, whether or not a big rock falls faster than
| a small rock [1] is a fairly basic question in physics,
| which has one of the most certain answers. Virtually
| nothing in psychology, let alone human social psychology,
| is at that level of certainty, and any psychologist worth
| their salt will agree with that. Basically any finding in
| psychology should have a level of certainty somewhere
| between "yeah, that's probably mostly true" and "hmm,
| interesting hypothesis that's not entirely crazy, I wonder
| if it holds up".
|
| [1] Also, the literal question of whether a big rock or a
| small rock falls faster is trickier than you might assume
| if you only know the middle school version of the question.
| If we're doing this on the moon, so as to dispose with the
| tricky aerodynamic questions, and we are answering the
| question from the frame of reference of an astronaut
| standing on the surface of the moon, the bigger rock
| actually does fall faster. Both rocks accelerate towards
| the center of the moon at the same rate, yes, but gravity
| works both ways. Both rocks pull on the moon but the bigger
| rock pulls harder, resulting in an extremely small torque
| on the lunar surface, drawing the side of the moon closer
| to it very slightly upwards. Which, from the surface frame
| of reference, is equivalent to the bigger rock falling
| faster.
| lupusreal wrote:
| The way you've put it, the "crackpot" and "reasonable"
| right wingers seem to have the same beliefs, but differ in
| their willingness to speak their mind or the ease with
| which they're bullied into silence.
| jdietrich wrote:
| It might have been surprising at the time, but it's certainly
| unsurprising in light of the replication crisis. We now know
| that most findings don't replicate. Anyone who continues to
| be surprised by _the most likely outcome_ needs to update
| their priors.
|
| https://en.wikipedia.org/wiki/Why_Most_Published_Research_Fi.
| ..
| neuronexmachina wrote:
| From your link:
|
| >Biostatisticians Jager and Leek criticized the model as
| being based on justifiable but arbitrary assumptions rather
| than empirical data, and did an investigation of their own
| which calculated that the false positive rate in biomedical
| studies was estimated to be around 14%, not over 50% as
| Ioannidis asserted.
| saurik wrote:
| FWIW, to me it is "unsurprising" as I can't remember the last
| time a major psych study actually did replicate, and it seems
| frankly like the entire field is in shambles. Does that mean
| that I _knew_ before that this study in specific was false?
| No. But, that isn 't what anyone means when they say
| "unsurprising"! If you get to the end of a movie and all of
| the major characters end up alive, and someone else seemed
| _shocked_ by that fact, you might still explain to them that
| that 's "unsurprising", even if you yourself got to feel the
| thrill of uncertainty -- or even were concerned for a bit --
| while you were watching.
| mistermann wrote:
| Be careful to not mistake "theory doesn't replicate" with
| "theory is not true [in whole or in part]".
| trhway wrote:
| Would you expect the theory of non replicative theories to
| be replicative or not?
| mistermann wrote:
| Expectations may be part of the problem.
| philwelch wrote:
| Stereotype threat was a "popular theory" because it fit the
| social and political presuppositions that were fashionable at
| the time. It's almost always wise to be skeptical of
| political fashions or new social science findings even on
| their own; new social science findings that match political
| fashions are doubly questionable.
| zahlman wrote:
| My understanding is that "growth mindset" represents a
| personal philosophy, not a testable hypothesis, so it would
| be irrelevant here.
|
| It was entirely reasonable to be skeptical of stereotype
| thread when the concept was new, _a priori_ , An
| "unsurprising" result is not necessarily one that someone
| confidently believed. If I flip a coin, I'm not "surprised"
| when the result is heads, nor when the result is tails.
| eslaught wrote:
| I could be wrong but I believe the idea is that the growth
| mindset has measurable consequences. I.e., if you believe
| you can grow, you will learn/adapt more effectively to new
| situations and ultimately achieve more. It's an empirically
| verifiable claim that is either true or false, and comes
| with a pretty straightforward intervention (i.e., teaching
| people that they can grow) if true.
|
| Of course people may also adopt it as a personal
| philosophy, but that's separate.
| zahlman wrote:
| Link for the claim that the results don't reproduce, without
| Facebook tracking: https://osf.io/preprints/psyarxiv/qctkp
|
| > When Black students at Stanford University were told that a
| test was diagnostic of intellectual ability, they performed worse
| than their white counterparts. However, when this stereotype
| threat was ostensibly removed--by simply framing the test as a
| measure of problem-solving rather than intelligence--the
| performance gap Black and white students nearly vanished.
|
| Just reading this description motivates me to reject the study
| out of hand. It's not plausible that university-level students
| responded meaningfully differently to being told "this is a test
| of problem-solving skill" versus "this is a test of intelligence"
| because _it is commonly understood that problem-solving skill is
| a major component of intelligence_.
|
| >it also became the darling of the political left who now had an
| answer to prevailing views of group differences held by the
| political right. This is partly because shortly before stereotype
| threat took its turn in the spotlight, Charles Murray and Richard
| Herrnstein published The Bell Curve... the octogenarian Murray is
| still considered a pariah, shouted down and deplatformed from
| talks he tries to deliver at respectable colleges to this day.
|
| The characterization of Murray's views in the last several years
| has been grossly uncharitable and seems entirely disconnected
| from his actual arguments. It's strange that the book is 30 years
| old, but has seemed politically relevant for much less time than
| that.
| aprilthird2021 wrote:
| Eh, I mean the stereotype being "activated" is that Black
| people are less intelligent / have lower IQs, not that they
| lack problem-solving skills.
|
| Even the replication attempt had two scenarios:
|
| 1 where women were told the test was to establish performance
| levels on the test between men and women
|
| 1 where they were told the test was a test of problem-solving
| skill (or primed to disregard negative stereotypes before the
| test).
|
| So even the replication has the incorrect framing you worried
| about. I tend to believe the problem wasn't this, but the way
| the field was lax about sampling, methodology, etc. After all,
| there were many stereotype threat studies, not just this one,
| boasting similar results. And they didn't all use that framing.
| andrewflnr wrote:
| Not to necessarily defend the study here, but I don't think
| that's a good reason to dismiss the study out of hand. Words
| themselves, the actual sounds independent of their technical
| meaning, still have very strong, deep-rooted associations that
| are hard to fight even if you've been educated. The subtext and
| day-to-day use of "intelligence" and "problem solving skill"
| are different, and color the way you think of them even if you
| know they're technically synonymous (which, do note, they're
| not quite). That association operates at the same deep level at
| which the supposed effect works.
| kens wrote:
| Many years ago, long before "The Bell Curve", I spent way, way
| too much time reading Murray's book "Losing Ground" (1984) and
| going to the library to read the papers he referenced and
| reading the papers those papers referenced and then the papers
| those papers referenced. My conclusion was that his book is
| junk. My second conclusion was that citations are like a game
| of telephone and every time someone cites something, it changes
| slightly, so don't trust anything until you get to a primary
| source, and maybe don't even trust that. (This second
| conclusion applies in general, not just to Murray.) My third
| conclusion was that arguing online is a waste of time and
| approximately nobody cares about actual facts.
| intalentive wrote:
| Your interlocutor might not care about facts but the audience
| might. I've learned interesting new things from reading the
| back-and-forth of others going at it online.
| runamuck wrote:
| Executive Summary of Article: "new data now reveal what many of
| us suspected for at least ten years: stereotype threat does not
| replicate, and it does not undermine academic performance in the
| ways we thought."
|
| The Stereotype Threat: "individuals who are part of a negatively
| stereotyped group can, in certain situations, experience anxiety
| about confirming those stereotypes, leading paradoxically to
| underperformance, thus confirming the disparaging stereotype."
| for example, if you remind a woman of the "Women are bad at math"
| stereotype, they will perform worse on a math test than if they
| are not reminded of that stereotype.
| verteu wrote:
| Also interesting: p28 (labeled p470) of
| https://journals.sagepub.com/doi/pdf/10.1177/251524591881022...
| ("Many Labs 2: Investigating Variation in Replicability Across
| Samples and Settings") shows the results of attempting to
| replicate 28 published psych results across many different
| samples.
|
| Unfortunately, "Stereotype Threat" is not one of the effects they
| attempted to replicate.
| underlipton wrote:
| Putting on my contrarian hat: the studies he mentions seem to be
| concerned with whether testing conditions affect stereotype
| threat effects. Logically, they can't prove or deny the
| phenomenon's existence, only whether testing conditions result in
| changes to test outcomes that could track with interventions to
| reduce stereotype threat. Much of what we know about how
| behavioral effects of identity comes with the understanding that
| the latter is something people carry with them, regardless of
| local or recent events. If there's a problem with stereotype
| threat as a concept, it's that it's positioned as a superficial
| effector that can be manipulated easily, rather than the surface
| level manifestation of complex interactions between self-
| identity, personal values, and cultural expectations. Based on
| the author's disclosure about his PhD thesis, he seems to be
| someone who capitalized on the former characterization, so of
| course he throws the baby out with the bathwater when it no
| longer works for those purposes. We might be looking at an ass-
| covering write-up.
|
| >Let's play "Find the Lebowski quotes game" again!
|
| So, yeah, I find this a deeply unserious blog post.
| bluGill wrote:
| He must throw the baby out with the bathwater - so far there is
| no evidence for anything here. The only alternative is to rerun
| the study, doing it correctly this time and then do a correct
| analysis. Until then you need to say "I don't know" when asked
| about this as that is all science allows. Maybe the rest of the
| baby they are throwing out will replicate and thus is correct -
| but nobody knows that and so we cannot say anything with
| confidence right now.
|
| I'm assuming you are not aware of studies not mentioned here
| that replicate - I'm not in this field and so I would not know
| where to look. I'm guessing that you also are not in this field
| and are looking for some way to allow your bias to become true
| despite these issues - but of course I might be wrong.
| underlipton wrote:
| Let me analogize. This is a blog post that claims that racism
| isn't real. As proof, the author explains that none of the
| studies that show a reduction in racism when you remind
| people that Michael Jordan and Beyonce exist can be
| replicated. Well, okay, sure. And the guy who earned a
| doctorate with the thesis, "Does Playing Jackson 5 Make
| People Less Racist?" might want to turn this into, "Racism
| does not exist," as a way to distance himself from his own
| terrible scholarship. (Also, did you find his Marvel
| Cinematic Universe Easter Eggs?)
|
| So, thus far, there is actually not enough evidence to throw
| the, "Racism exists," baby out with the, "Do these
| interventions affect racist belief?" bathwater. And since
| we're second-guessing biases, it's super weird that you're
| always in comment sections of (politically-charged) articles
| concerning fields you're not in.
| buttercraft wrote:
| > This is a blog post that claims that racism isn't real.
|
| No, it does not make that claim.
| shermantanktop wrote:
| I think that contrarian hat may include a no-true-scotsman
| badge.
|
| If stereotype threat is real, we should be able to have a
| replicatable study result that confirms it, right? We're not
| just limited to logical inference, I hope.
| underlipton wrote:
| If the studies are concerned with determining that stereotype
| threat exists, sure.
| whimsicalism wrote:
| > Let's be honest: that last sentence was far too generous. Many
| of us engaged in practices that, in hindsight, were borderline
| dishonest. We abused experimenter degrees of freedom, engaged in
| questionable research practices, p-hacked, massaged our data--you
| pick the euphemism. In contrast, this new replication study
| followed the most up-to-date best practices in psychological
| science, eliminating room for flexibility in analysis or results
| interpretation.
|
| Exploiting researcher degrees of freedom remains unfortunately
| extremely common. There needs to be some sort of statistical
| vanguard in the ivory towers enforcing _real_ preregistration and
| good analysis practices. Strict epistemic discipline is necessary
| to do real science.
| bluGill wrote:
| The real problem is we have a lot of data. So it is easy to
| shove it all into a computer and then see what comes out. This
| is not a bad thing, but it isn't alone a result to publish it
| is instead justification to do a real study on whatever thing
| statistics turns up as a significant result to make sure it
| really is when you properly isolate for that data.
| parpfish wrote:
| There's also the issue that in psych, everything is measured
| through indirect proxies and most of the predictions will be
| directional. That gives experimenters a LOT of wiggle room.
|
| In physical science you can often directly measure a
| phenomenon and have a theory that makes very specific
| predictions
| parpfish wrote:
| early in my psych career, i was actively _encouraged_ to do
| things that would be considered p-hacking or abusing degrees of
| freedom. it was seen as a way to not let any data go to waste
| and potentially 'show off' your cleverness/mathematical chops
| in the methods section (anybody can just run an ANOVA like they
| teach in the text book, but _look what I can do_ ).
| taeric wrote:
| I'm curious what the impact for most people's mindsets will be
| here? I'm imagining that it relates to what many people consider
| "talent" with kids? I know that has been an odd trend in some
| states trying to get rid of "gifted" programs and such. I have
| largely remained hopeful that that was not nearly as prevalent as
| online represents.
| akoboldfrying wrote:
| >this study (still a preprint) was preregistered (meaning all
| methods and analyses were specified before the data were
| collected)
|
| This is the way forward -- preregistered studies. That, together
| with a promise from the publisher to publish the result
| _regardless of whether the effect is found to be significant._
|
| When you think about it, the incentives for publishing in science
| have been wrong all along. The future will be different: It will
| be full of null results, of ideas people had that didn't pan out.
| But we'll be able to trust those results.
| ckemere wrote:
| I also have a high degree of skepticism about most psychology
| research. I find it frustrating that the author of this piece
| raises the issue of the imbalance of gender in STEM jobs as a
| reason to doubt this particular study. There is nothing about
| this failure-to-replicate that should allow us to conclude that
| innate differences in math ability underly that imbalance.
|
| I'd love others to read the replication report and explain why I
| might be wrong?
| ckemere wrote:
| I mean come on - if there is a pervasive cultural insinuation
| of gender differences in ability that leads to women
| experiencing more test anxiety (I'm not saying there is, but
| conditioning on that hypothesis) why would we expect a 20
| minute PowerPoint to counteract 20 years of people's life
| experience!?!?
| anthuswilliams wrote:
| The author doesn't claim that the imbalance is a reason to
| doubt the study. His claim is that the imbalance is a reason to
| doubt the claim that an underlying shift in cultural norms
| explains why the results might have been valid in 2005 but
| failed to replicate in 2024.
|
| The author rightly observes that despite the undeniable shift,
| women are still significantly underrepresented in STEM and
| therefore that cannot explain the lack of replication. There
| are still many other reasons besides innate differences that
| might explain it.
| armoredkitten wrote:
| As someone who went through grad studies in the field, and who
| has met Dr. Inzlicht in person before, I have to say I deeply
| appreciate his perspective. He has consistently been humble when
| facing the issues in the field, in ways that call even his own
| previous research (following the typical practices of the day)
| into question. The field as a whole has been undergoing a
| reckoning, but Mickey has been one of the people who has
| encouraged his fellow researchers not just to wag their fingers
| at others, but also to look inward and reflect on their own
| research practices. He has done so by showing humility and
| acknowledging where his research has fallen short, and that
| indicates to me a great deal of integrity.
|
| It is sad to see stereotype threat being one of those findings
| that seems less and less credible. I once worked as a research
| assistant on a project related to stereotype threat, and I recall
| the study going through several iterations because it all needed
| to be _just so_ -- we were testing stereotypes related to women
| and math, but the effect was expected to be strongest for women
| who were actually good at math, so it had to be a test that would
| be difficult enough to challenge them, but not so challenging
| that we would end up with a floor effect where no one succeeds.
| In hindsight, it 's so easy to see the rationale of "oh, well we
| didn't find an effect because the test wasn't hard enough, so
| let's throw it out and try again" being a tool for p-hacking,
| file drawer effects, etc. But at the time...it seemed completely
| normal. Because it was.
|
| I'm no longer in the field, but it is genuinely heartening that
| the field is heading toward more rigour, more attempts to correct
| the statistical and methodological mistakes, rather than digging
| in one's heels and prioritizing theory over evidence. But it's a
| long road, especially when trying to go back and validate past
| findings in the literature.
| simpaticoder wrote:
| _> It is sad to see stereotype threat being one of those
| findings that seems less and less credible._
|
| While I'm sure it is an honest statement, this sentiment is
| itself concerning. Science is ideally done at a remove - you
| cannot let yourself _want_ any particular outcome. Desire for
| an outcome is the beginning of the path to academic dishonesty.
| The self-restraint required to accept an unwanted answer is
| perhaps THE most important selection criteria for minting new
| academics, apart from basic competency. (Acadmeia also has a
| special, and difficult, responsibility to resist broader
| cultural trends that seep into a field demanding certain
| outcomes.)
| parpfish wrote:
| I don't think there's anything wrong with wanting a
| particular outcome. It wasn't wrong that people were excited
| about the prospect of a room temp superconductor a few months
| back because people knew that if it were true good things
| were possible. Insisting that you can't be excited by one
| potential outcome from a study means that you'll only study
| things that don't have the potential to help
| simpaticoder wrote:
| There is a useful distinction between "wanting" and
| "attachment", but one usually turns into the other. Your
| mention of room temp superconductors is ironic since they
| have all been precisely attachment-driven frauds that start
| with wanting.
| freehorse wrote:
| These are the exceptions imo. Most of it is funding-
| driven (or sometimes wanting-status-driven) than pure
| wanting-driven. Most researchers do not even care that
| much about their actual field and would change the field
| to do what they really want to do, albeit funding keeps
| there where they are.
| disgruntledphd2 wrote:
| > While I'm sure it is an honest statement, this sentiment is
| itself concerning. Science is ideally done at a remove - you
| cannot let yourself want any particular outcome.
|
| This basically never happens. I worked in academia for many
| years, and in psychology for some of that, and I have never
| met a disinterested scientist.
|
| Like, you need to pick your topics, and the research designs
| within that etc, and people don't pick things that they don't
| care about.
|
| This is why (particularly in social/medical/people sciences)
| blinding is incredibly important to produce better results.
|
| > The self-restraint required to accept an unwanted answer is
| perhaps THE most important selection criteria for minting new
| academics,
|
| I agree with this, but the trouble is that this is not what
| is currently selected for.
|
| I once replicated (four times!) a finding seriously contrary
| to accepted wisdom and I basically couldn't get it published
| honestly. I was told to pretend that I had looked for this
| effect on purpose, and provide some theory around why it
| could be true. I think that was the point where I realised
| academia wasn't for me.
|
| Now, the same thing happens in the private sector, but
| ironically enough, it's much less common.
| jrmg wrote:
| I have seen stereotype threat mentioned in educational contexts -
| justifying the idea that, for example, it's important not to make
| (deliberately or inadvertently) things like coding or engineering
| seem stereotypically masculine. Usually the recommendations are
| to ensure that if pictures of participants in classes or
| extracurricular programs are shown in advertising, diverse groups
| of people are pictured - or that if connections to popular
| culture are made in educational materials they're diverse - for
| example, don't make all your example coding projects about Star
| Wars or football.
|
| Anecdotally, I've seen with my own eyes, for example, girls
| getting really into coding only after seeing it demonstrated by
| enthusiastic women that they can see as role models in ways they
| would not see men.
|
| I guess this is a far broader thing than stereotype threat - but
| I'm sure this larger thing is real. I fear that people who
| themselves have stereotypes in mind about who 'should' be into
| certain topics will use the demise or deemphasis of stereotype
| threat to justify not making attempts to attract or be friendly
| to kids who really could flourish in non-stereotypical fields -
| to their and society's detriment.
| vacuity wrote:
| Yep, I agree that this kind of descriptive representation, role
| model thing is very real and can have significant effects. It's
| the best kind of evidence that someone of the same demographic
| likely can/can't achieve similar things.
| disconap wrote:
| I participated as a subject in a research study at Stanford
| involving race and stereotype threat in the early 2000s. The
| details are hazy, but the final readout was the distance I put my
| chair to a group of chairs that students of a particular racial
| group were supposed to sit. Evidently I put them in a position
| that was contrary to the effect the researcher was seeking. She
| intensely asked me a ton of questions about my background and
| eventually tossed my data point for having lived in a racially
| diverse area growing up. This wasn't a pre-inclusion criteria,
| but a possible act of scientific fraud. Huge bummer since there
| are honest people in every profession, and I imagine a lot of
| them didn't succeed the way that the fraudsters thrived.
| ndileas wrote:
| I've had a similar experience in a long running survey. When I
| gave the "wrong" answers, the interviewer asked a bunch of
| questions and eventually told me to skip certain questions.
| mitthrowaway2 wrote:
| I had an experience like that where an education researcher
| was attempting to prove that a certain style of teaching was
| more effective. It involved a test measuring how much
| students remembered from a conventionally-taught course taken
| the year before; the researchers hoped to show that these
| scores were low, and therefore that conventional teaching
| methods were ineffective. I and another classmate aced the
| test so the researcher accused us of cheating, even though
| there was no incentive to cheat (the test wasn't used for any
| grade, so if anything, the incentive was to save time by
| leaving answers blank). We denied cheating, explained that
| the course instruction had been very memorable, and proved it
| by correctly answering followup questions on the spot.
| Ultimately our high-scoring test results were discarded as
| outliers and the hypothesis was successfully validated.
| freehorse wrote:
| > a possible act of scientific fraud
|
| It sounds more like another day in vegas for the psychology
| field of that era. This was not an exception, and often
| researchers were not even aware they were doing something
| wrong. Even nowadays psychology researchers are clueless as to
| what really p-hacking and bad statistical practices mean. And
| because they consider themselves honest researchers, while
| these practices are obviously dishonest, they do not consider
| themselves actually doing anything like that - but somebody
| else may! it is always somebody else. Now it is not as bad as
| that period, and there is more awareness about the most blunt
| violations of statistical rigour, but the actual understanding
| is still low for the median researcher so many grey-to-black
| zones exist still.
| greentxt wrote:
| Incentives determine outcomes. Most grad students are grad
| students because they are responding to incentives. It's
| competitive so cheating is ubiquituous. Reduce competition in
| academia and problems would lessen.
| bjourne wrote:
| Usually psychology studies are conducted by field workers or
| assistants. I don't think the person you interacted with was
| the actual author of the study.
| freehorse wrote:
| Grad students and post-docs are often running studies.
| Animats wrote:
| This is partly the "psychology is the study of college
| undergrads" problem. The original study was on Stanford students,
| all of whom have already passed through a very selective filter.
| That's not a group to extrapolate to the general population. Too
| many psych studies are done on this convenient population.
|
| It's great seeing the original author admit the problem.
| istjohn wrote:
| No it's not. This is a completely different thing.
| parpfish wrote:
| one of my stances has been that psychology comes up with wacky
| unreplicable findings because there's no central organizing
| theories for how 'the mind' works and everything is just very
| black-box.
|
| there are so many studies showing "X manipulation affects Y
| outcome", but there's not even a hint of an attempt to explain
| the mechanisms in a meaningful way (cognitive experiments are
| usually better, but often still guilty of this).
| disgruntledphd2 wrote:
| I actually disagree entirely. The amount of forcing theories
| that are well ahead of the facts that occurs in academic
| psychology is off the charts.
|
| We could do with a shut up and experiment movement, to be
| honest.
|
| In order to build useful theories, we need lots more data, and
| forcing theories onto data actively holds us back from running
| the wide variety of experiments necessary for a real theory to
| emerge.
| parpfish wrote:
| I agree that there isn't as much raw experimentation and just
| observing/documenting psychological phenomena that there
| should be. I've ranted about that here in the past.
|
| I guess my peeve is that nobody really collaborates on
| theories. Every PI is off working on their own set of pet
| theories (because that's how you establish a career), but
| there's really no centralizing force to get people to
| collaborate and work on a shared theory. Physics has the
| standard model that everyone can use as a common reference
| point and it's helpful.
| gotoeleven wrote:
| When I first heard about this stuff 20 years ago, it was being
| presented as "The differences between groups A and B go away when
| stereotype threat is removed" which is not what the original
| paper says. The original paper claims that whatever difference
| there is between groups A and B, it will be larger when measured
| under the condition of stereotype threat which I guess is
| plausible but much less interesting.
|
| Part of the context of that time was that _The Bell Curve_ had
| been published fairly recently and there was great desire to
| disprove it and anyone doing that could count on lots of
| attention and speaking fees. So the grift was to present
| stereotype threat as this grand solution that could resolve all
| racial differences.
| foxbarrington wrote:
| I was a psych major in undergrad, and did an experiment as a riff
| on stereotype threat and got a small effect. I had the
| participants solve brain teaser puzzles and the only difference
| was introducing them as coming from 11th grade or graduate level
| math. Undergrads did worse when they thought it was graduate
| level.
| rozab wrote:
| I think that result would ring true for anyone who's worked in
| education, but it doesn't sound very similar to stereotype
| threat
___________________________________________________________________
(page generated 2024-12-21 18:01 UTC)