[HN Gopher] Fabricated data in research about honesty
___________________________________________________________________
Fabricated data in research about honesty
Author : danso
Score : 91 points
Date : 2023-07-28 15:04 UTC (7 hours ago)
(HTM) web link (www.npr.org)
(TXT) w3m dump (www.npr.org)
| cscheid wrote:
| I'm a former academic who did a bit of work that involved
| statistical analysis of data in a similar way as these studies.
|
| Fuck. These. People.
|
| We put so much work making sure the analysis is right. Sweat the
| design, sweat the data collection, the consent forms, and dropped
| the studies where the result wasn't there, as well as reporting
| null results. I specifically remember having to have a terrible
| conversation with a phd student that, after doing a power
| analysis, setting up a pilot study, and collecting enough data,
| we got p=0.054 on the relevant test. "Sorry, we can't add
| additional participants; we can't try a new analysis. We did the
| right thing and we should report a non significant result. It
| sucks."
|
| And these fuckers just go ahead and forge data to land TED talks?
|
| Yes, let's also have a conversation about incentives and the
| system. But individually, I reserve my sympathy for better
| people. Salt the earth they walk on, and rake them over coils.
| This is beyond reprehensible. Fuck. Them.
| nyokodo wrote:
| > This is beyond reprehensible.
|
| It is easy enough to make honest mistakes and spread
| misinformation before it can be properly replicated that we do
| still require scientists to be honest. Active deception should
| come with a high cost. There should be a scientific equivalent
| of disbarment.
| cscheid wrote:
| > Active deception should come with a high cost
|
| Yes, I wish so too. For what's worth, I'd buy "either of Dan
| Ariely and Francesca Gino write a NYT bestseller book in 5
| years, go on a redemption tour, and nothing new happens" for
| $0.80 on a prediction market.
| jvanderbot wrote:
| This is the latest in my most frequently learned lesson:
|
| Nobody _thinks_ they are doing anything wrong. They think they
| are shoring up a weakness in a fundamentally good argument or
| cause.
| godelski wrote:
| While I generally go with this, it definitely isn't an
| absolutely true statement and we should even modify it. People
| know they are doing wrong, they just justify it with reasons
| why it is okay to do the wrong thing. I'm pretty sure that
| these people knew they were doing a big wrong, but I'm sure
| they justified it with "well everyone fudges the data a
| little." (this wasn't little)
| bena wrote:
| I'm going to go with both of you.
|
| They know what they're doing is wrong. But I don't think they
| did it because "everyone does it", they did it because they
| think they're ultimately right.
|
| They believe in their results, they know in their heart of
| hearts that their conclusions are true. It's just that not
| all of the data lines up. It's incomplete, inconclusive, etc.
| Yes, it is wrong to fabricate data, however, over time,
| results will bear out the truth of their statements and it
| won't matter.
|
| They are of the camp that process doesn't matter as long as
| you reach the right conclusion. Which is wrong. Process is
| kind of the only thing that matters.
| AnimalMuppet wrote:
| Ouch. That's not just scientific dishonesty. It's also
| politics, and the legal system, and advertising, and internet
| conversations, and...
| verisimi wrote:
| "the greater good"
|
| "THE GREATER GOOD"
|
| https://www.youtube.com/watch?v=5u8vd_YNbTw
| munchler wrote:
| Yep. There are very few true villains in reality. Just a lot of
| people with different values and priorities. Even Vlad Putin
| probably thinks he's on the side of the angels.
| xdennis wrote:
| Please don't call that guy Vlad. Vlad is short for Vladislav.
| You're looking vor Vova.
|
| It's like saying Ben is short for Bennett.
| tomrod wrote:
| Ben can be short for Bennett?
| functoid wrote:
| [flagged]
| dang wrote:
| Related. Others?
|
| _Fraudulent data raise questions about superstar honesty
| researcher (2021)_ -
| https://news.ycombinator.com/item?id=36726485 - July 2023 (33
| comments)
|
| _UCLA professor refuses to cover for Dan Ariely in issue of data
| provenance_ - https://news.ycombinator.com/item?id=36684242 -
| July 2023 (131 comments)
|
| _Harvard ethics professor allegedly fabricated multiple studies_
| - https://news.ycombinator.com/item?id=36665247 - July 2023 (215
| comments)
|
| _Harvard dishonesty expert accused of dishonesty_ -
| https://news.ycombinator.com/item?id=36424090 - June 2023 (201
| comments)
|
| _Noted study in psychology fails to replicate, crumbles with
| evidence of fraud_ -
| https://news.ycombinator.com/item?id=28264097 - Aug 2021 (102
| comments)
|
| _A Big Study About Honesty Turns Out to Be Based on Fake Data_ -
| https://news.ycombinator.com/item?id=28257860 - Aug 2021 (90
| comments)
|
| _Evidence of fraud in an influential field experiment about
| dishonesty_ - https://news.ycombinator.com/item?id=28210642 - Aug
| 2021 (51 comments)
| impalallama wrote:
| > The two worked together on a paper about how to "nudge" people
| to be more honest on things like forms or tax returns. Their
| trick: move the location where people attest that they have
| filled in a form honestly from the bottom of the form to the top.
|
| Another strike against the entire concept of "nudges". Its become
| clear that it was a bit of a liberal pipe dream saying you could
| accomplish huge changes in people or even society with
| imperceptible "nudges" that conveniently meant you didn't need to
| address systemic issues and stone wall any attempts at grander
| change.
|
| "Be more honest by changing a form." "Make people less fat by
| moving where the produce is in the store." Turns out things are
| more complicated.
| sogen wrote:
| In my experience doing research, this question should go *at
| the bottom*.
|
| Why: A basic 101 thing one learns in Market Research is that
| personal questions _always_ go last.
|
| So, if you want to ask an interviewed person if their answers
| were honest, researchers should be placing them last, towards
| the end of the questionnaire.
|
| But, this only works if the questionnaire is oriented towards
| asking first impersonal questions, then more personal ones.
|
| As another commented said: This is a lot of work. Doing
| research, asking people questions, gathering and then analyzing
| data is no walk in the park. It's sad that this happened. And,
| most important of all: Always being honest, no matter what the
| results are, even if you don't like them.
| jstx1 wrote:
| > conveniently meant you didn't need to address systemic issues
| and stone wall any attempts at grander change
|
| Who is claiming this?
| impalallama wrote:
| https://en.wikipedia.org/wiki/Nudge_theory
| [deleted]
| gruez wrote:
| >Its become clear that it was a bit of a liberal pipe dream
| saying [...] that conveniently meant you didn't need to address
| systemic issues and stone wall any attempts at grander change.
|
| Really? I thought liberals were all about addressing systemic
| issues (eg. systemic racism), and all about "grander change"
| (eg. defund the police/ICE, eat the rich, etc.).
| lo_zamoyski wrote:
| The word "liberal" is terribly overloaded to mean all sorts
| of things. The OP's intended meaning seems to be liberal in
| the sense of a lack of coercive measures, preferring
| incentives instead. Your meaning seems to mean "leftist".
|
| FWIW, I use it to describe that which is Lockean in substance
| (which entails a kind of radical, atomistic individualism
| that takes the individual to be the basic unit of society,
| and not the family), which basically means that most of the
| political spectrum is implicated.
| mmanfrin wrote:
| > a liberal pipe dream
|
| Non sequitur political flamebait in the top comment. HN
| continues its slide.
| Shugarl wrote:
| > Make people less fat by moving where the produce is in the
| store
|
| It doesn't? Not even a little bit ? Genuine question. One of
| the example of data exploitation I was given in university is
| that retail companies look for patterns in the thing their
| customers buy, and when they see that people who buy X-kind of
| thing also tend to buy Y-kind of thing, they tend to put X and
| Y right next to each other to push the customers to buy X and
| Y. Wouldn't doing the opposite work ?
| impalallama wrote:
| Moving single digit volume of sales and "solving" obesity are
| two orders of magnitude different things.
| Calavar wrote:
| They may be referring to the research of Brian Wansink, who
| was exposed a few years ago for at the very least shoddy
| research practices and at worst active data manipulation [1,
| 2, 3]. A large body of his work centered around how to
| rearrange school lunchrooms to promote better dietary
| decisions. His advice quite literally includes placing
| healthy food first [4].
|
| [1] https://www.wsj.com/articles/a-cornell-scientists-
| downfall-1...
|
| [2] https://web.archive.org/web/20180307074049/http://www.tim
| van...
|
| [3] https://www.buzzfeednews.com/article/stephaniemlee/who-
| reall...
|
| [4] http://www.brianwansink.com/for-school-lunches.html
| [deleted]
| tantalor wrote:
| > bit of a liberal pipe dream
|
| Fail to see any connection here.
| rgrieselhuber wrote:
| I believe it's a reference to these initiatives:
|
| https://www.politico.com/agenda/story/2015/10/obamas-
| effort-...
| godelski wrote:
| This article is talking about things like defaulting
| options to organ donor or automatically enrolling people
| into a 401k plan. I'm not sure changing the default option
| is the same as changing someone's behavior. Sounds like
| it's about explicitly NOT changing the behavior but rather
| the outcome of that behavior.
|
| Something you may notice about these fraud cases is that
| they are simple things that result in _surprising_
| outcomes. I don't think people are surprised that texting
| teens works better given that the only people that call
| them are parents and telemarketers. There's two fundamental
| lines in the article too
|
| > government has long been behind corporate America in
| following up to make sure its policies actually work.
|
| > The SBST team's report showed that 13 of its projects
| worked; one didn't; and one was hard to tell.
|
| And if we're getting down to it, this doesn't sound like an
| Obama or "liberal" thing, it sounds like a _government_
| thing. Can we stop taking general concepts or universal
| behavior and pretending that when its bad only "my enemies"
| do it and when good only "my allies" do it? This is just
| dumb, and it happens even outside the context of politics.
| rgrieselhuber wrote:
| > And if we're getting down to it, this doesn't sound
| like an Obama or "liberal" thing, it sounds like a
| _government_ thing. Can we stop taking general concepts
| or universal behavior and pretending that when its bad
| only "my enemies" do it and when good only "my allies" do
| it? This is just dumb, and it happens even outside the
| context of politics.
|
| I'm all for this, although it does require an agreement
| on epistemic principles, which seems like a tall order in
| the current state of our civilization. Which is why we
| are left with will to power.
| sudosysgen wrote:
| Looking for solutions that do not involve any structural
| change is inherent to liberal politics in our society, since
| our current political economy is deeply liberal in structure
| (for a definition of liberal that contains both mainstream
| "liberals" and conservatives, closer to classical
| liberalism).
|
| You can see this plainly by looking at the pattern of
| recuperation and dilution of any proposed structural change -
| making up non-structural alternatives that don't work but
| might and then haranguing over them for years is a common
| pattern in liberal democracies. See for example trying to
| avoid public healthcare by creating "optimized marketplaces",
| the entire idea of nudging in economics, the idea of self-
| regulation, avoiding providing free national tertiary
| education by instead giving progressively stringent and oddly
| targeted scholarships, etc...
| shanusmagnus wrote:
| Good paper on this topic:
|
| https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4046264
|
| Chater, N., & Loewenstein, G. (2022). The i-frame and the
| s-frame: How focusing on individual-level solutions has led
| behavioral public policy astray. Behavioral and Brain Sciences,
| 1-60. https://doi.org/10.1017/S0140525X22002023
| thimkerbell wrote:
| Are there countries where behavioral public policy has not
| been led astray? (I have not read the article)
| sproketboy wrote:
| [dead]
| whynotmaybe wrote:
| Kinda ironic that my most remembered detail about Dan Ariely's
| book is that "everyone cheats a little".
|
| https://www.wired.com/2009/02/ted-1/
| apienx wrote:
| Recommended video on the matter: https://youtu.be/d2Tm3Yx4HWI
| ldehaan wrote:
| [dead]
| boberoni wrote:
| The podcast "If Books Could Kill" has a great episode that breaks
| down and debunks the book "Nudge". The podcast also factchecks
| various other pop-science and so called "airport books".
| gloryjulio wrote:
| I am glad that with internet I could cut out those word salad
| crap as much as possible, and stick with more reputable and
| updated sources.
|
| Basically we have a better package management system to reduce
| the supply chain attack
| firewolf34 wrote:
| Now we just have to apply that concept to the watering-down
| which occurs when they try to take white-papers and
| disseminate them to the public through news articles and etc.
|
| It is important for the public to learn - and they will need
| assistance to understand findings that are outside of their
| usual realm of knowledge/experience. Unfortunately, the
| resources for this are shaky, profit-motivated, and
| arbitrary.
| verisimi wrote:
| 'debunks Nudge?'
|
| Someone should tell the politicians and PR consultants...
| rahimnathwani wrote:
| The two linked source posts are excellent:
|
| http://datacolada.org/109
|
| http://datacolada.org/98
|
| This latest one has great sleuthing examples:
|
| http://datacolada.org/112
|
| Their analysis suggested that the data (originally collected
| using Qualtrics) had been modified, i.e. that the data in the
| data files would not match the original data (still stored on
| Qualtrics' servers).
|
| It turned out they were right: We have received
| confirmation, from outside of Harvard, that Harvard's
| investigators did look at the original Qualtrics data file and
| that the data had been modified.
___________________________________________________________________
(page generated 2023-07-28 23:02 UTC)