[HN Gopher] How a middle aged CS major debunked a classic positi...
___________________________________________________________________
How a middle aged CS major debunked a classic positive psychology
finding (2013)
Author : jvanderbot
Score : 277 points
Date : 2022-02-07 16:16 UTC (6 hours ago)
(HTM) web link (narratively.com)
(TXT) w3m dump (narratively.com)
| MetaWhirledPeas wrote:
| I see this as an argument for adding more STEM to _every_ degree
| plan. There 's unavoidable crossover, and people need intuition
| to properly question unfounded claims.
| wbsss4412 wrote:
| Unfortunately I'm not sure that STEM in and of itself is the
| antidote.
|
| It's very easy to just routinely apply methods without properly
| reasoning about the underlying assumptions.
| FredPret wrote:
| But you shouldn't be able to get a degree in a soft "science"
| without demonstrating that you can apply disciplined thinking
| by passing some proper STEM courses.
| wbsss4412 wrote:
| Nominally, that is already the case.
| klabb3 wrote:
| Be careful what you wish for. Increasing the competence of
| fraudsters will simply make fraud harder to detect.
|
| Had the original theory simply included a "measured"
| coefficient, the debunking would have been much weaker and
| never made it into a gotcha story for bystanders to enjoy.
| This was sloppy from the outset.
|
| What we need isn't stronger technical skills, but strong
| self-correction mechanisms and incentive structures.
|
| (Ironically, the management consultant who orchestrated this
| whole ploy is in the field of group psychology - so perhaps
| it was performance art all along)
| MetaWhirledPeas wrote:
| The less exposure one has to these concepts the more tempting
| it becomes to skip the reasoning part.
| wbsss4412 wrote:
| True, but there is an upper bound to the amount of exposure
| one can receive before they are simply a STEM major, rather
| than another major with more STEM exposure.
| Domenic_S wrote:
| > _Additionally, Fredrickson references a textbook called
| Differential Equations, Dynamical Systems, and an Introduction
| to Chaos, a book Sokal says "is aimed at upper-level
| undergraduates in mathematics, physics and related
| disciplines."_
|
| > _"If she had studied 1 /10th of what she said she studied,"
| says Sokal, "she would have seen right through Losada."_
|
| The author making the bad claims seems to have a pretty decent
| math education. Or she's lying about what she studied...
| MetaWhirledPeas wrote:
| The real people in need of STEM were all those _reading_ the
| original paper. Those seeking to be published could
| substitute ill intent for ignorance.
| pvarangot wrote:
| I think if you add more STEM people will use mathematical
| models incorrectly even more because they'll feel more
| confident about understanding the tools they don't.
| SeanLuke wrote:
| Isn't psychology part of STEM?
| Quikinterp wrote:
| It's a social science so no I don't think most people would
| consider it part of STEM.
| YaBomm wrote:
| duxup wrote:
| It seems weird this theory's backbone was just some random person
| who contacted someone and said something like "hey I have math
| that says you're right" and it just takes off from there. Until
| some other rando checks it out of curiosity...
| consumer451 wrote:
| I think you have just described the open source code review
| model.
|
| _edit: and science in general?_
| duxup wrote:
| -PR approved "Rando said it looks good "-
| wheelinsupial wrote:
| Do you mean this quote:
|
| "A Chilean business-consultant named Marcial Losada, 'who had
| begun to dabble in what had become his passion: mathematical
| modeling of group behavior,' sent her an out-of-the-blue
| email."
|
| The article seems to ignore it, but Losada had a PhD in social
| and organizational psychology from the U of Michigan [1].
|
| [1] http://losada.socialpsychology.org/
| ChrisMarshallNY wrote:
| I've been watching _Dopesick_ , and the part, where they are
| chasing down the "1%" study, is pretty damn sobering.
| klabb3 wrote:
| Citations are upvotes for academia.
| jvanderbot wrote:
| TL;DR
|
| Nick Brown was a 50y/o CS/IT employee who wanted to cross-train
| to human resources and wanted "evidence-based stuff". He
| encountered lots of fantastic positive psychology claims that
| were being accepted wholesale as "quantitative". He traced the
| math behind a critical one back to a computational fluid dynamics
| paper, which the debunkees had used equations from with no
| context. (allegedly because of the use of "Flow" in positive
| psychology!)
|
| He cold emailed professors asking for feedback, wrote a 3000 word
| critique, got help from a mathematician and psychologist in
| smoothing the writeup, and ultimately got it published in a top
| journal and the offending results recognized as flawed. The
| general crusade continues along with co-crusaders Sokal and
| Friedman.
|
| The result in question was the legendary 3-1 positive to negative
| tipping point between flourishing individuals and languishing
| ones.
| newsbinator wrote:
| A key bit I pulled out:
|
| > It seemed a case of numbers fudging. In a valid fluid
| dynamics problem, the numbers plugged into the equation must
| correlate to the properties of the fluid being studied. But in
| attempting to draw an equivalence between the physical flow of
| liquids and the emotional "flow" of human beings, Losada had
| simply lifted the numbers that Lorenz used in 1963 to explain
| his method in the abstract, numbers used merely for
| illustrative purposes. Losada's results, along with the pretty
| butterfly graphs Brown had been shown in class, were
| essentially meaningless.
| readthenotes1 wrote:
| TL;DR: Most psych research results are garbage. Some make
| millions for the authors. Here's yet another example.
| Taylor_OD wrote:
| Some make millions for Authors who take those papers and cite
| them in popular psych books. The actual authors of the papers
| are rarely, if ever, making millions.
| frostburg wrote:
| The 9 million dollars in grants that the author received
| probably were also completely wasted, since someone that
| would randomly apply CFD equations to people is unlikely to
| have published worthwhile research.
| CRConrad wrote:
| Remains to be tallied how many of her research assistants
| on those projects went on to have careers in the field...
| Careers which are now more or less tainted by the
| association.
|
| Even harder to tally: How many of those deserve to be
| more tainted because they saw it for what it was but
| didn't speak up. (Yeah, I know, hard to do for a lowly
| assistant.)
| 0xcde4c3db wrote:
| I don't know whether it made millions, but Fredrickson did
| write a pop-psych book based on the critical positivity
| ratio: _Positivity: Top-Notch Research Reveals the 3-to-1
| Ratio That Will Change Your Life_.
| klabb3 wrote:
| "Top-Notch" is the giveaway.
| errcorrectcode wrote:
| Toxic positivity and toxic negativity are grating to some, myself
| included.
|
| The issue is that soft, squishy not-STEM "sciences" aren't all
| that evidence-based or experimentally-verifiable. That doesn't
| make them completely useless, it makes them prone to fungible,
| inconsistent guidelines rather than hypotheses, theories, or
| laws.
| phkahler wrote:
| It doesnt seem possible to have a fixed positivity ratio when
| considering research on extinction.
|
| The gist of extinction is that when training using positive
| feedback, the desired response will cease when the positive
| feedback does (extinction). In order to maintain the behavior the
| feedback has to essentially be randomized with a decreasing
| prevalence (positivity ratio).
| davidwritesbugs wrote:
| The whole "Cited 350 times" made me laugh. I used to think this
| type of number sounded impressive. Recently I just started to do
| a Masters and was told "1 citation for every hundred words", so
| dozens of papers need reading and it take for *ever* weeks to
| read just a small number. Then I recalled articles claiming most
| citations are never checked so, and you won't believe this, I
| skim read papers and if vaguely relevent inserted cursory
| comments from the paper and cited it. Did that go horribly wrong?
| Go on, guess. So 350? I so believe it.
| 8K832d7tNmiQ wrote:
| Ah yes, the classic Goodhart's law [0] working as intended
| again.
|
| [0]:https://en.m.wikipedia.org/wiki/Goodhart%27s_law
| ALittleLight wrote:
| That seems like it's own separate problem which is also bad.
| davidwritesbugs wrote:
| It was just a lazy metric to measure effort and engagement,
| and that relieved the lecturer of the need to read the paper
| himself to see if you'd absorbed it and analysed its
| relevance correctly. I can't say I'm unsympathetic - the guy
| had 50 students to mark. I'm guessing this is standard in
| academia but if they'd said "read and include relevant
| papers" I'd have done a good job on half a dozen and the
| lecturer probably wouldn't have been overburdened either.
| dang wrote:
| I remember past threads about this but can't find them. Anyone?
|
| Edit: the list so far - thanks caaqil and setgree! I feel like
| there were still others, if anyone can find them...
|
| _Nick Brown Smelled Bull (2013)_ -
| https://news.ycombinator.com/item?id=9791302 - June 2015 (1
| comment)
|
| _The British amateur who debunked the mathematics of happiness
| (2014)_ - https://news.ycombinator.com/item?id=16515822 - March
| 2018 (213 comments)
|
| _The British amateur who debunked the mathematics of happiness
| (2014)_ - https://news.ycombinator.com/item?id=9085639 - Feb 2015
| (15 comments)
| setgree wrote:
| inter alia: https://news.ycombinator.com/item?id=16515822
| caaqil wrote:
| https://news.ycombinator.com/item?id=9085639
| froh wrote:
| And it's a long detailed read about the genesis of this paper:
|
| The Complex Dynamics of Wishful Thinking
| https://pubmed.ncbi.nlm.nih.gov/23855896/
| throwawaypt wrote:
| Anyone have suggestions of the best way to call BS on clinical
| studies and their marketing? A publicly traded company's products
| are all based on BS science. I've pointed it out to them but they
| are dead silent. Just one example is them comparing (in
| marketing) the active group of one study to the control group of
| another study performed in different countries years apart with
| different inclusions criteria massively different n and insanely
| different dropout rates. It just blows my mind this stuff
| happens.
|
| If anyone is interested here is the full writeup:
|
| https://www.sendbig.com/view-files?Id=c131d2fc-9a4a-0616-f77...
| georgecmu wrote:
| For those curious: The two nonsense papers are:
|
| _Losada, Marcial. "The complex dynamics of high performance
| teams." Mathematical and Computer Modelling 30 (1999): 179-192._
| | https://www.sciencedirect.com/science/article/pii/S089571779...
|
| _Fredrickson, B. L., & Losada, M. F. (2005). Positive Affect and
| the Complex Dynamics of Human Flourishing. American Psychologist,
| 60(7), 678-686._ |
| https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3126111/
|
| The debunking paper is
|
| _Brown, N. J. L., Sokal, A. D., & Friedman, H. L. (2013). The
| complex dynamics of wishful thinking: The critical positivity
| ratio. American Psychologist, 68(9), 801-813._ |
| https://arxiv.org/pdf/1307.7006.pdf
| monksy wrote:
| I can guarantee you there are a lot of people really upset with
| him.
|
| A lot of those kinds of studies are "this makes me feel good and
| this is the world I want to see" types. No consideration of the
| negative outcomes of bad info, bad practices, or even
| externalizes.
| [deleted]
| j7ake wrote:
| A quote struck me as relevant during covid and work from home
| times:
|
| "because more good things will happen to you than bad on any
| given day, but nothing will happen to you if you just sit
| indoors."
| f154hfds wrote:
| Interesting read! The most disturbing part is not fraud
| scientists, I think there will always be a certain percentage of
| fraudsters in any field. The most disturbing part is the 300+
| citations, scientists who based their work off of an obviously
| garbage paper.
|
| It seems inescapable that the reason why must have been complete
| mathematical incompetency across the entire field. Say a
| psychologist partners with perhaps a chemist and uses some
| organic chemistry to 'prove' some critical point in a paper. I
| could imagine brushing over the details and citing the paper when
| any chemist would see the fraud as 'obvious'. Such is the
| complexity of cross discipline science I guess, and the reality
| that the sum total of human knowledge is beyond any individual's
| ability to grasp.
|
| That being said, knowing that some papers are utter garbage I
| should do my due diligence by taking a humble pill and reaching
| out to my colleague in the chem department across the street to
| tell me what this critical chemistry is about before I include it
| in my work.
| justin66 wrote:
| > The most disturbing part is the 300+ citations, scientists
| who based their work off of an obviously garbage paper.
|
| > It seems inescapable that the reason why must have been
| complete mathematical incompetency across the entire field.
|
| I think you won't have to try very hard to find analogous
| failures to properly test the truth or falsehood of the
| literature's conventional wisdom in more "mathematically
| competent" fields.
| TameAntelope wrote:
| I think this is a fantastic example of how a) anyone can call
| bullshit on science if it doesn't make sense and b) there _are_
| more and less correct ways of calling bullshit on established
| science.
|
| You don't need to hop on a podcast or radio show in order to
| question the orthodoxy of a scientific claim, and in fact you
| shouldn't if you want to be taken seriously. Nick Brown would
| have hurt his case had he done that, and instead found a path
| that _actually_ knocked down a faulty concept in a way that has
| lasting effects.
|
| This is the process working. This is all okay, and welcome, IMO,
| though possibly not by everyone involved.
| pasquinelli wrote:
| that's an odd interpretation. this seems less like a system
| doing what it's supposed to do and correcting itself and more
| like an instance of that system failing to do what it's
| supposed to do and correcting itself.
| nostromo wrote:
| The people that publish and promote bullshit science spend a
| lot of time on podcasts, so I don't see why the people
| falsifying their research shouldn't as well.
| bonoboTP wrote:
| > This is the process working. This is all okay, and welcome
|
| No! This sort of thing casts enormous doubt on the field of
| psychology and many soft sciences as a whole. A lot of it is
| academic politics and people being afraid of calling things out
| in fear of hurting their career prospects. It's much easier to
| just stay silent, ignore it, and play along. Who knows who will
| sit on your next grant proposal's committee. Academia is full
| of this and it's a disgrace.
| FredPret wrote:
| I, for one, will never trust another "finding" coming out of
| the field of psychology.
|
| Between this and the marshmallow study, I think psychology
| should be considered an art. We're many centuries away from
| this progressing to being a science on par with physics.
|
| There's nothing wrong with that, a wise therapist has much to
| contribute - but not to science.
| lrem wrote:
| Marshmallow study? Is the finding about "wait 5 minutes to
| get two sweets - success in life" debunked, or are you
| referring to another thing?
| quag wrote:
| Yes, it has been debunked. They got the causal arrow
| around the wrong way.
|
| It turns out that kids who are in stable situations with
| trustworthy adults around them, tend to trust adults more
| when they say "wait 5 minutes and I'll give you more",
| and that kids in bad situations don't trust adults. Turns
| out being in a bad situation as a kid does have a long
| term effect on your life.
|
| Nothing to do with the ability of people delay
| gratification.
| lotsofpulp wrote:
| This xkcd is relevant:
|
| https://xkcd.com/435/
|
| Unless the variables in the experiment are decently
| quantifiable. It is a garbage in, garbage out situation.
| FredPret wrote:
| The problem with psychology is: the subject matter is the
| most complex system we know of.
|
| In order to make definitive statements about said system,
| you need to start with the most basic behaviours and
| characteristics.
|
| As you work your way up, you'll mountains of data to
| capture every permutation of human behaviour and emotion.
|
| We're so, so far from any of that.
| naasking wrote:
| The messier the system, the more rigorous the methodology
| should be. Pre-registering trials to fight p-hacking, and
| open data should be the norm. That's decidedly not what
| we've gotten from this field, and some prominent figures
| have even been fighting such measures and consider the
| reproducibility crisis to not be a big deal. That's the
| problem.
| ramblenode wrote:
| More often than not the constructs in the garbage studies
| are quantifiable, just useless. The bigger problem is the
| "what" of what is being measured---whether that thing is
| representative, useful, or important. Part of the reason
| bad research has the veneer of authenticity is that the
| numbers, graphs, and statistics dress up the nonsense.
| lotsofpulp wrote:
| Yes, that is the distinction I was trying to make by
| writing "decently quantifiable".
| TameAntelope wrote:
| If what you've read here concerns you to the point of
| throwing out the field, you're going to be _immensely_
| disappointed by every other scientific field, as well. [0]
| I 'm not an expert, but my understanding is that this
| article could have been written about literally any field
| of science, with a depressingly few nouns changed. [1]
|
| This is what people mean when they say, "science is messy".
| This is how the sausage gets made, and if you're cool
| throwing out psychology as a field, you're probably also
| going to have to be cool throwing out a _lot_ of other
| fields. [2]
|
| There are efforts being made to clean up how science is
| conducted, called "metascience", but it's a relatively new
| field and there's still a lot that hasn't been investigated
| yet. [3]
|
| Basically, if you've ever thought of yourself as "pro
| science", you've been supporting what happened in this
| article, whether you realized it or not.
|
| [0] https://www.nature.com/articles/533452a
|
| [1] https://www.scientificamerican.com/video/is-there-a-
| reproduc... -- a slightly more accessible version of [0]
|
| [2]
| https://direct.mit.edu/posc/article/28/4/482/97500/Mess-
| in-S...
|
| [3] https://en.wikipedia.org/wiki/Metascience
| fhood wrote:
| I think we can maybe inject a little bit of extra nuance
| here.
|
| Not every field is the same, they vary massively in size,
| level of external influence, and complexity in subject
| matter. Psychology suffers more than most. It's a large
| field, so competition to get noticed is likely high.
| There is money to be made, as discussed in the article,
| so it struggles with influences outside of pure
| scientific inquiry, and they have no chance of describing
| their theories in terms of pure component mathematics.
|
| Contrast that with ocean and atmospheric sciences. It is
| a tiny and very insular field, so the rat race works very
| differently. This also protects them from outside
| influences, and a lot of the major underlying physical
| and biological processes involved can be modeled with
| mathematics that have been demonstrated to be predictive.
|
| It's just easier for some fields to reliably produce
| higher quality work. I think this is a case where blanket
| skepticism is just lazy. Every field suffers some level
| of a reproducibility issue, but some fields are better
| able to self select for a variety of reasons.
| ModernMech wrote:
| Thank you for saying this. I mean, Computer Science is up
| there in the junk science department. Coming from a
| phsyics background, it's hard to even call most of what
| happens in CS "science".
|
| Go to any CS conference and you'll see a whole bunch of
| papers doing micro benchmarks, saying that their method
| is X% faster than all the others, where X% is usually
| some very low percentage (and of course no errors bars
| are visible on any plots). You talk with the guy
| presenting the research and you find out that half the
| benchmarks were done on one machine, while the other half
| were done on another with different specs. Well, okay,
| now why can I trust the benchmarks? And how many _other_
| papers are doing this? And is your result even meaningful
| anymore? It 's so depressing.
|
| And don't even get me started on the AI/ML field. My god,
| talk about art over science.
| bonoboTP wrote:
| A lot of AI/ML work is sloppy but well known methods work
| reliably, textbook descriptions make sense. That there's
| a long tail of shitty student papers doesn't invalidate
| the field. Year after year methods genuinely get better.
| Whatever process produces this progress, it produces this
| progress.
| bee_rider wrote:
| There's an unfortunate tendency to treat science as the
| only source of truth. Really, psychology (like engineering
| and medicine) is a profession. Professions aren't just
| based on the scientific method -- they draw from a semi-
| formal pile of experience known as "best practices." These
| are often just based on something that a clever person
| decided to try decades or centuries ago which seems to have
| not hurt anyone. Of course it is an iterative process so
| sometimes these things are overturned, but the process
| seems to work allright.
| naasking wrote:
| How could you possibly know that the "process seems to
| work alright" without scientific evidence that it's doing
| so? You actually have no idea in that case whether you're
| doing harm or helping, and if you're somehow helping, it
| could just be a very expensive placebo.
| bee_rider wrote:
| I mean, I don't work in mental health, but if somebody
| has mental health issues that they are dealing with, and
| they buy an "expensive placebo" in the form of talking
| about their problems with a professional, and it helps...
| that seems like a successful treatment, right? I'm
| actually not even sure how to define placebo in this
| context.
| naasking wrote:
| I think it's important to focus on "expensive" and not
| "placebo" in that statement. If a placebo is all that's
| needed (or all that we can do), then arguably it
| shouldn't be expensive.
| bakuninsbart wrote:
| I think the differentiation between "science" and
| "profession" is a bit crude. You obviously don't include
| mathematics in this, but it is a form of deductive
| reasoning as far away from science as the arts are.
| Deductive reasoning is just far less prone to fault than
| inductive reasoning, be it mathematics or philosophy, and
| the methods we have developed in some of the natural
| sciences over the centuries to model real-life behavior
| on mathematics are really great.
|
| You are right that these largely don't exist in the
| Social sciences, but this is at least partially due to
| the much more complex subject matter. Still, both the
| social sciences as well as the natural sciences are
| trying to approximate reality through models built on
| experiments, and are thus fundamentally different from
| deductive reasoning in closed systems.
|
| Not trying to mince words here, but engineering is
| applying validated knowledge to solve real world
| problems, sure, you can call it profession, but the
| fields generally associated with engineering are still in
| the business of generating knowledge. Medicine and
| Psychology heavily rely on the scientific method to
| validate abstract concepts, while engineering adjacent
| disciplines like Computer Science heavily really on maths
| and deductive reasoning to solve problems in the space of
| algorithms and computers.
| tremon wrote:
| _You obviously don 't include mathematics in this, but it
| is a form of deductive reasoning as far away from science
| as the arts are._
|
| I disagree strongly with this. To reduce mathematics (and
| philosophy in your next sentence) to not-a-science, just
| because it's based on deductive reasoning is doing it a
| major disservice.
|
| It is especially because of its deductiveness that
| mathematics is such a valuable tool for science. It
| answers the fundamental question "assuming A, what can we
| prove to be true about A' and A''?". Without that
| deductive proof, many inductive scientific theories could
| not even be formulated, let alone disproven.
|
| And then you go on stating that _engineering is applying
| validated knowledge to solve real world problems_. Do you
| realize that much of that validation of said knowledge
| has been done through mathematical models? That the only
| reason we have certainty in engineering is because of the
| deductive rigour of mathematics?
| FredPret wrote:
| Mathematics & philosophy aren't sciences because there
| are no hypotheses, experiments, and results; it's just
| thinking and logic.
|
| It's still incredibly valuable and the basis of many good
| things.
|
| I would also add that engineering isn't only working with
| existing scientific knowledge; we were building steam
| trains and optimizing them long before we started doing
| scientific thermodynamics.
| bumby wrote:
| There was an interesting discussion on HN last week that
| brought up the fact that mathematics has its own crises,
| namely a communication crises. It was brought up that the
| proofs can be so dense that errors will be published and
| go unchecked for a long time. The interesting part is
| that everything that's needed for "replication" is
| literally within the paper and the errors still fall
| through the cracks.
|
| There is still an awful lot of engineering that is model-
| based without strong deductive proofs. (Mechanical
| failure theory comes to mind). But the actual origin of
| "profession" comes from professing an oath to the public
| good. Meaning the traditional professions (law, medicine,
| engineering) aren't necessarily aimed at creating new
| knowledge, but applying knowledge to the betterment (or
| profit) of society. Sometimes that butts up against
| problems without known solutions that requires adding
| knowledge to the field, but that's not really the primary
| goal, unlike something like mathematics.
| FredPret wrote:
| Spot on. I've met many psychologists who struck me as
| kind, wise people with a helpful perspective. Society
| needs them.
|
| But to plug this fuzzy, professional knowledge into
| equations, call it science, and use it to make decisions
| in business and policy is at best naive and at worst
| fraud.
| endgame wrote:
| The article describes events set into motion over a decade ago.
| People calling bullshit on things that happened over the past
| couple of years have had a much worse time of it.
| hex4def6 wrote:
| I'm not sure I agree. In his own words, he had to ship around
| for a respected authority so that it would be taken seriously.
|
| Once he had that, the journal have a bullshit response that it
| was passed two months since the original article was published,
| and so not eligible for publication. Sokal through his
| connections, went to the CEO to get that decision over ruled
| (edit: by threatening that the journal would be humiliated by
| refusing to publish it). The original debunker would never have
| got that far on his own.
|
| To me, this sounds like a process that is far from some sort of
| egalitarian "judge ideas on their merit rather than the person
| presenting them." If he hasn't got Sokal involved, I wonder if
| this would have gotten published at all.
|
| Edit: also, this casts a pretty negative light on the field,
| given that it's been cited 350+ times in this field. Not one of
| those other papers dug into the content (or understood it)
| enough to question it, and/or raise the alarm bells. It's one
| thing Its one thing if this was some obscure paper that never
| got eyes on it, but that is obviously not the case.
| jokethrowaway wrote:
| Science can only be as good as the people doing it.
|
| The problem is that most papers, especially on very
| complicated topics, are not science and just serve the
| purpose of proving a point or helping someone's career.
|
| Even without taking easy jabs at psychology (you would need
| full brain understanding of its inner workings and evolution
| over a lifetime, to have a 100% clear picture of what's
| happening - we probably have 0.0001% - it's all based on
| observations and interpretations), the reproducibility crisis
| is affecting a lot of areas.
|
| Hopefully with more time and more people getting involved in
| the field, we'll get more and more science.
|
| If I think about nutrition / fitness, between 40-20 years ago
| and now we made great progress: we went from "Fats are bad
| and make you fat" (pure propaganda to sell processed sugars),
| "BULK on a 4000 diet" to any video of Jeff Nippard quoting
| incredibly detailed studies about minutiae of nutrition and
| building muscle.
| [deleted]
| mindcrime wrote:
| _I 'm not sure I agree. In his own words, he had to ship
| around for a respected authority so that it would be taken
| seriously._
|
| I don't see how that contradicts the person you are replying
| to. Ze is saying there is _a_ path to contradicting
| established science. Ze didn 't say that it was easy, or that
| it didn't involve working _with_ people who are in "the
| establishment" - just that jumping on podcasts and talk shows
| and spouting off as an outsider with no credibility _isn 't_
| the best way.
| native_samples wrote:
| That probably depends on your goals.
|
| If your goal is to try and reform from within then sure,
| spending months waiting for BS replies from journals is the
| best/only way.
|
| If your goal is to spread the word that psychology is
| unreliable and you don't care much about what psychologists
| themselves think, then podcasts and talk shows are a far
| more effective strategy.
|
| Psychology isn't really a very harmful field - bad claims
| in psychology mean maybe some people waste time on a
| useless self help method, or don't get proper psychiatry
| when maybe they should. But a lot of low level damage still
| adds up, and there's a lot of people who listen to
| psychologists. If you go the media (NB: which is what Brown
| is doing here), then you can potentially have a more
| positive impact on balance.
| pasquinelli wrote:
| is it a path or is this case a fluke? that's the question.
| dandelany wrote:
| I agree completely with criticisms of the journal - clearly
| their process is flawed in a multitude of ways. But regarding
| "shopping around for a respected authority to be taken
| seriously" - I think this is actually an important lesson.
|
| The problem with the vision of an egalitarian meritocracy in
| science (or any other field, honestly) is that reviewing,
| understanding and critiquing someone else's work has a non-
| zero cost - actually quite a large cost if done correctly.
| Also, the people most qualified to do this analysis generally
| loathe doing it, as they'd rather be spending time on their
| own original work. Inevitably, people develop heuristics to
| estimate the credibility of every new email or article that
| comes across their desk, to decide which is even worth
| reading.
|
| Understandably, one of those heuristics is
| credentials/authority - the ratio of good articles published
| by non-experts in their field is extremely small and
| massively outweighed by crackpots. So what does a smart non-
| expert with a good idea do to hack this system? "Bootstrap"
| your authority, by first finding someone with more
| credentials than yourself, but not so much notoriety that
| they're too busy to read your email. If you can convince
| them, they'll bring it up the chain to the next-highest
| authority _they_ can convince, and eventually it 'll have
| some important names behind it and people will pay attention.
| Brown was able to accomplish this with only two
| collaborators, so I consider him a master bootstrapper for
| this :)
|
| Everyone wants to be the patent clerk slaving away in
| isolation who pops out with a new revolutionary theory of the
| universe, but it's just not realistic these days. On the
| internet, a non-expert trying to rapidly gain credibility on
| their own is indistinguishable from social media
| hucksterism/"influencing". If you want to break into a new
| field, find some experts in that field and work with them!
| mcguire wrote:
| " _Everyone wants to be the patent clerk slaving away in
| isolation who pops out with a new revolutionary theory of
| the universe, but it 's just not realistic these days._"
|
| And note that that patent clerk wasn't _just_ some patent
| clerk. He had published four papers (plus a bunch of
| reviews) and finished his dissertation by 1905.
| Strs2FillMyDrms wrote:
| Juxtaposing this to Nakamoto's creation of bitcoin and
| original use cryptography, it makes no sense to me how
| anyone would have ever paid even an ounce of attention to
| what some random, supposedly unkown guy on the internet has
| to say about ... anything. Or how Perelman even got someone
| to even eant to verify his solution. Maybe not so much
| Perelman since he had some important figures knowing him
| already.
| dandelany wrote:
| Yeah Bitcoin is such an interesting exception! I think
| _some_ ideas are viral enough on their own to go from
| zero to front-page on HN /reddit/etc. even when posted by
| an anonymous user - but it's rare to go beyond that, they
| usually run out of momentum.
|
| Bitcoin had a few things going for it - the whitepaper is
| concise and well-explained, in perfect academic prose and
| typeset in LaTEX, which lends it an air of credibility.
| It came with an implementation, so hackers could
| experiment with it immediately, plus it sort of has
| implicit libertarian anti-government undertones, both of
| which encouraged follow-on blog posts.
|
| Once the ball got rolling, another kind of heuristic took
| over - the money heuristic. People put money into the
| thing, which made other people go "oh look at how much
| money they put in, this is a real serious investment
| thing!" and put in _their_ money, in a sick feedback loop
| that continues to this day...
| dfabulich wrote:
| Bitcoin received early support from high-reputation folks
| in the cryptography community, including Hal Finney and
| Nick Szabo. (As a result, some people thought that one or
| both of them _were_ Nakamoto.)
|
| Regardless of who Nakamoto actually was, it does appear
| that Finney and Szabo played a similar role that Sokal
| did in this story; getting support from high-rep folks
| was essential to getting Bitcoin off the ground.
| inglor_cz wrote:
| Perelman was widely known as a math prodigy (he won gold
| for the USSR at the International Mathematical Olympiad)
| and studied at the Leningrad University, so he could
| approach pretty much any of his former professors to take
| a look. He was definitely not an outsider in the field. A
| recluse, yes, but that is different from being a no-name.
| kurthr wrote:
| The journal publishers (similar to FB et al) are commercial
| entities designed to make money from desperate academics
| trying to publish. They thrive on novelty and readership, but
| also "respect" (rather than ads and "engagement") from not
| having their occasionally outrageous papers questioned.
| Sometimes to stoke or avoid scandal and increase readership
| they might include a debunking paper, but it's important to
| remember these are not bought by individuals but by
| departments who won't stock "less respected" journals.
|
| Now, if perhaps there were a respected public psychology
| paper service like psyarXive at the time that might have made
| things slightly better.
|
| https://psyarxiv.com/
| jvanderbot wrote:
| I agree. The real shining light here is that a determined
| amateur was able to get _free, volunteer help_ from _inside the
| community_ to weather the publication process.
|
| Those who read the review / revision process in this article
| and were dismayed / convinced the process was flawed should
| realize this is precisely how review works in many fields. It
| takes a year to get a paper reviewed and revised, and a "25
| page" (TFA) statement of revision is par. Not "all" will have
| the stomach for that, but once you are part of that process,
| it's normal.
| horsawlarway wrote:
| You consider having to threaten the reputation of an academic
| journal by emailing the CEO directly before you _even make it
| to the review_ for a well written, concise, critical paper
| "part of that process"?
|
| Not even mentioning that it required the publicity and
| reputation of a well known figure (Sokal) to even open _that_
| door for them.
|
| If that's the "shining light" of this industry - I sure as
| fuck don't want to see the dirty alleyway.
| jvanderbot wrote:
| I didn't say that.
|
| I said the shining light was insiders helping.
|
| I also said the peer review process is challenging and that
| was an accurate representation.
|
| Two different things.
| tdrgabi wrote:
| I understand what you mean, but I see a series of lucky
| moments, that all had to happen for the paper to be published.
|
| Sokal picked it up, even if usually he doesn't. The journal
| rejected it and they had to extra push it etc.
|
| You can see how many rebutals, done without podcasts are never
| heard of when going through the oficial path.
| gjvc wrote:
| _I understand what you mean, but I see a series of lucky
| moments, that all had to happen for the paper to be
| published._
|
| The role of luck is often under-estimated.
| CRConrad wrote:
| > The role of luck is often under-estimated.
|
| Then so is the role of _bad_ luck.
|
| Betcha there are more papers that don't have the luck this
| one did than there are that do, donchathink?
| 300bps wrote:
| Wikipedia has a good tl;dr:
|
| https://en.wikipedia.org/wiki/Critical_positivity_ratio
|
| _Later in 2013, the critical positivity ratio aroused the
| skepticism of Nick Brown, a graduate student in applied positive
| psychology, who felt that the paper 's mathematical claims
| underlying the critical positivity ratio were fundamentally
| flawed.[6] Brown collaborated with physicist Alan Sokal and
| psychologist Harris Friedman on a re-analysis of the paper's
| data. They argued that Fredrickson and Losada's paper contained
| "numerous fundamental conceptual and mathematical errors", as did
| Losada's earlier work on positive psychology, which completely
| invalidated their claims.[7] In their response, Hamalainen and
| colleagues argued that there were no fundamental errors in the
| mathematics itself, but the problems relate to the interpretation
| and justification of the use of the model.[8] Losada declined to
| respond to the criticism, indicating that he was too busy running
| his consulting business.[6] Fredrickson wrote a response in which
| she conceded that the mathematical aspects of the critical
| positivity ratio were "questionable" and that she had "neither
| the expertise nor the insight" to defend them, but she maintained
| that the empirical evidence was solid.[9] Brown and colleagues,
| whose response was published the next year, maintain that there
| is no evidence for the critical positivity ratio.[10]_
|
| _In response, American Psychologist formally retracted the
| mathematical modeling elements of Fredrickson & Losada's paper,
| including the specific critical positivity ratio of 2.9013, as
| invalid._
| Joeboy wrote:
| Since TFA is quite long and has quite an annoying layout, I
| tried Reader View in Firefox, and it gave me a completely
| different article. Weird. Thanks for posting the tl;dr.
| CRConrad wrote:
| Yup, same here. I think it was because it's one of those
| "infinite scrolling" pages, where when you've scrolled down
| to the end of the article another begins (and the URL in the
| address bar has also magically changed).
|
| Yet another reason to hate that shit.
| mannerheim wrote:
| 'I think the educational and psychological studies I mentioned
| are examples of what I would like to call Cargo Cult Science. In
| the South Seas there is a Cargo Cult of people. During the war
| they saw airplanes land with lots of good materials, and they
| want the same thing to happen now. So they've arranged to make
| things like runways, to put fires along the sides of the runways,
| to make a wooden hut for a man to sit in, with two wooden pieces
| on his head like headphones and bars of bamboo sticking out like
| antennas--he's the controller--and they wait for the airplanes to
| land. They're doing everything right. The form is perfect. It
| looks exactly the way it looked before. But it doesn't work. No
| airplanes land. So I call these things Cargo Cult Science,
| because they follow all the apparent precepts and forms of
| scientific investigation, but they're missing something
| essential, because the planes don't land.' - Richard Feynman,
| 'Cargo Cult Science'
|
| https://calteches.library.caltech.edu/51/2/CargoCult.htm
| renewiltord wrote:
| His Twitter account: https://twitter.com/sTeamTraen
|
| Seems he does a lot of this data policing stuff. Neat.
|
| The funny thing is that a lot of people go around saying "yeah,
| engineers go around thinking they can just walk into field X and
| study it without context". Yup. They do, and they can.
|
| Another article
| https://www.theguardian.com/science/2014/jan/19/mathematics-...
| dogleash wrote:
| >that a lot of people go around saying "yeah, engineers go
| around thinking they can just walk into field X and study it
| without context".
|
| That's sorta the point of a degree in the sciences rather than
| going to a vocational school, right?
|
| It's amusing that people find it arrogant. It is that they're
| showing a level of competence we're all claiming and maybe only
| pretending to have? Is it "swim in your own lane" union-style
| isolation? Is it sneering at nerds for not having the good
| sense to shut it? Is it inflated egos that still think the
| things they learned in undergrad were hard?
| at-fates-hands wrote:
| >> The funny thing is that a lot of people go around saying
| "yeah, engineers go around thinking they can just walk into
| field X and study it without context"
|
| I have a degree in Anthropology and transitioned into UX
| Research. 90%+ of what you do is observation in both fields.
| Most of the engineers and math people I've met are horrendously
| bad at observing behavior.
|
| In some fields, its just a different set of skills engineers
| are not capable of applying very well.
| derbOac wrote:
| Sometimes they can and sometimes they can't. Works like that
| for all kinds of crossovers, and probably should happen more
| often than it does.
|
| I can think of plenty of times CS has benefitted from
| psychology too, or is reinventing the wheel.
| renewiltord wrote:
| Haha, yes. I'm just a big fan of people trying. I'm also a
| big fan of anti-gatekeeping.
| derbOac wrote:
| I surely agree with you on that.
| [deleted]
| colechristensen wrote:
| I got bored. This is an article style I really don't like, the
| interesting information is buried towards the end after thousands
| of mostly irrelevant bits of narrative which aren't worth
| remembering.
|
| I can guess the ending though, technical guy sees psychologists
| appropriating a well defined mathematical structure, doing some
| hand waving, and claiming that it applies to your everyday life
| in a shallow way. Guy is hero.
|
| Yes, psychology is absolutely full of bad science and bad
| scientists. Lots of fields are, but particularly psychology.
| p_j_w wrote:
| >This is an article style I really don't like, the interesting
| information is buried towards the end after thousands of mostly
| irrelevant bits of narrative which aren't worth remembering.
|
| This sort of journalism is something I find extremely
| frustrating. Honestly, I don't care about pretty much anything
| in the entire section about Brown meeting Wiseman. That whole
| thing could've been a single paragraph with no valuable
| information lost. Maybe this is just because we're in the ADHD
| age, but I couldn't read it because of all of this extraneous
| information.
| colechristensen wrote:
| This isn't an instant gratification issue, the details which
| pad this article just aren't very relevant to the point or
| the title. The life story and mundane details of this man's
| life don't, or barely, support the main point of the article
| which is how he debunked and got retracted a well-known paper
| in a field which wasn't originally his. I might want to read
| a biography of Albert Einstein, but the biography of this guy
| just isn't remarkable.
|
| It's the same issue with SEO recipe articles where before the
| actual recipe there's an inane story about going to the beach
| with the family dog and finding a starfish before finally
| getting to the point about fish tacos, or whatever. If the
| article had been fine details about a few of the ingredients
| or sourcing them or preparation techniques down to specifics,
| sure, that would make sense; but if I ask you what time it is
| and you start telling me about how your grandfather served in
| WWII I'll probably be a little annoyed.
| MaxBarraclough wrote:
| Agreed. You're not the only one who dislikes having their
| time wasted by that kind of writing.
|
| A bit masochistic, but here's my 'favourite' example of this
| kind of writing, shamefully from none other than the BBC. [0]
|
| See how long it takes you figure out what on Earth they're on
| about. Also, see how long it takes you to realise the title
| is an outright lie.
|
| [0] https://www.bbc.co.uk/news/world-australia-49012771
| TameAntelope wrote:
| I'm glad I'm not the only one who felt this article was about
| twice as long as it needed to be.
|
| I will throw out there that, despite all of the flaws and
| issues with Psychology as a field of science, it does seem like
| everyone involved, mostly, did the right thing here. I did skim
| it, but I didn't really see anyone do anything other than have
| sour grapes over how it went down, so I feel like this article
| and situation generally is a good example of a flawed system
| working out; deferring to truth over ego and orthodoxy.
|
| Some groups would have circled the wagons to protect against
| this outsider, but that's not what ultimately happened here.
| Good!
| [deleted]
| BeefWellington wrote:
| > Yes, psychology is absolutely full of bad science and bad
| scientists. Lots of fields are, but particularly psychology.
|
| Social sciences in general are probably harder than other
| fields for the simple reason that it requires other people's
| time and effort.
|
| A psychologist friend of mine once dropped the line "we know a
| lot about how university students view the world" when
| discussing the replication crisis. I think that while
| simplifying things quite a bit, it is very apt.
|
| _Edited for coffee_
| tgv wrote:
| If the latter were true, I'd be surprised. It's obviously a
| quip about using students as subjects for credits, but the
| average psychology study reveals very little about students.
| After all, it's how they got convinced psi exists (Bem,
| 2011).
| mywittyname wrote:
| Economics is where you really glean insights into
| university students. For example, they enjoy free food
| exponentially more than they like very cheap food. Also,
| they great at playing games for money.
| colechristensen wrote:
| It's more than that, people who aren't really interested in
| science or the math required to do good science are attracted
| to the "social sciences" in a much higher degree. People who
| are unqualified to do analyses are publishing papers and
| those papers are getting published with the approval of their
| peers who are similarly likely to be unqualified.
| colechristensen wrote:
| A related concept which is just a bit broader is WEIRD:
| https://en.wikipedia.org/wiki/Psychology#WEIRD_bias
|
| Western, Educated, Industrialized, Rich, and Democratic
| dominate the participants in psychology studies, even when
| they're not just psychology undergrads encouraged to
| participate in research.
| Scarblac wrote:
| This is a good quote:
|
| > "One could describe a team's interactions as 'sparky' and
| confidently predict that their emotions would be subject to the
| same laws that govern the dielectric breakdown of air under the
| influence of an electric field. Alternatively, the interactions
| of a team of researchers whose journal articles are characterized
| by 'smoke and mirrors' could be modeled using the physics of
| airborne particulate combustion residues, combined in some way
| with classical optics"
| derbOac wrote:
| To be honest I think this story is overplayed a bit. I've heard
| of that tipping point idea before, or the positive to negative
| ratio thing, but I think most in the field didn't take it more
| seriously than an observation that couples, groups, etc that
| function well are more positive and supportive.
|
| It's good to call this stuff out but part of the reason no one
| else did is because outside of a certain subsubfield no one
| took it that seriously.
| naasking wrote:
| The founders of that field got tens of millions of dollars
| for continuing that bunk research. You don't think that
| funding could have been used better?
| FabHK wrote:
| Overplayed? It jeopardises the entire edifice of that
| subfield, and by extension psychology itself, if it is not
| vigorously disowned.
|
| Sokal said it well (from the article):
|
| > "For me, the real question is not about Fredrickson or
| Losada or Seligman," Sokal says. "It's about the whole
| community. Why is it that no one before Nick--and I mean Nick
| was a first semester part-time Master's student, at, let's be
| honest, a fairly obscure university in London who has no
| particular training in mathematics--why is it that no one
| realized this stuff was bullshit? Where were all the supposed
| experts?"
|
| > "Is it really true that no one saw through this," he asks,
| "in an article that was cited 350 times, in a field which
| touts itself as being so _scientific?"_
| munificent wrote:
| I strongly disagree with this take.
|
| The whole point of science is to acquire knowledge that
| eventually gets communicated outside of the scientific body
| producing it. If the papers never leave the ivory tower, or
| never lead to other papers that do, it's just navel gazing.
|
| When a scientific community has a chunk of their work that
| everyone in the tower knows is bullshit but never gets
| corrected, _that metadata doesn 't leave the ivory tower_.
| What happens instead is that papers like this flutter out the
| window, get picked up by journalists who don't have access to
| tribal knowlege, and then get spread wide and far. The
| information appears legitimate because it _did_ come from a
| trusted scientific body. It just had the necessary context
| that it 's actually bullshit left off.
|
| Every scientific community has an obligation to ensure that
| the papers they publish and advocate actually mean what they
| say they mean. If nothing else, because not doing that
| ultimately tarnishes the brand and undermines the
| trustworthiness of that community.
| CRConrad wrote:
| > Every scientific community has an obligation to ensure
| that the papers they publish and advocate actually mean
| what they say they mean. If nothing else, because not doing
| that ultimately tarnishes the brand and undermines the
| trustworthiness of that community.
|
| And in the end of science as a whole.
| robwwilliams wrote:
| You may well be right, but this is too casual an approach
| toward what purports to be science in a nominally good
| journal. Modern science is amazingly well insulated from "in-
| house" criticism after the review process. As a result crap
| can flourish. We need more data/analyst wasps.
| photochemsyn wrote:
| Taking equations from physics out of context to bolster social
| science claims isn't unique to psychology, other social sciences
| like economics have done the same thing with little more
| justification. The rationale behind this approach is simple
| marketing, in that 'using more mathy stuff' will sell better to
| the general public than non-mathy hand-waving claims will.
|
| However, the practioners of this approach don't seem to
| understand that physical equations are constructed by analysis of
| empirical experimental and observational data on one hand, and
| theoretical mathematical development on the other, within limited
| scopes of applicability. Copying equations from one area of
| science to another willy-nilly like the article describes is just
| ridiculous - would cosmologists studying general relatively just
| plug in quantum mechanical equations in the hope of getting a
| 'mathy explanation'?
|
| Here's another example from 'mathematically sound neoclassical
| economics', which does the same with 19th century physical
| science equations:
|
| > "The strategy the economists used was as simple as it was
| absurd--they substituted economic variables for physical ones.
| Utility (a measure of economic well-being) took the place of
| energy; the sum of utility and expenditure replaced potential and
| kinetic energy. A number of well-known mathematicians and
| physicists told the economists that there was absolutely no basis
| for making these substitutions. But the economists ignored such
| criticisms and proceeded to claim that they had transformed their
| field of study into a rigorously mathematical scientific
| discipline."
|
| https://www.scientificamerican.com/article/the-economist-has...
| notahacker wrote:
| There are niche heterodox subfields of economics that literally
| are just a dubious physics analogy (thermoeconomics), but the
| source's argument that marginal utility theory was copying von
| Hemholtz is the sort of extraordinary claim that requires
| extraordinary evidence, not zero citations, baffling confusion
| of economist Francis Edgeworth with literary writer Maria
| Edgeworth and a lot of tacked on claims that are laughably
| false (Jevons certainly didn't believe that "natural resources
| are inexhaustible: he was actually initially famous for trying
| to predict when the coal supply would run out!).
|
| There are obvious reasons for studies of how money is spent to
| involve mathematics (though Menger managed to deduce marginal
| utility theory from first principles _without_ equations) and
| it 's unsurprising some of the equation forms superficially
| resemble some of the many simple equations of physics. There
| are much better criticisms of neoclassical economics and the
| application of neoclassical economics than accusing the early
| economists of plagiarising another field!
| ulucs wrote:
| >Because neoclassical economics does not even acknowledge the
| costs of environmental problems and the limits to economic
| growth
|
| Econophysics is really crank stuff but some people really like
| to shit on econ without knowing a tenth of the discourse in the
| field. How can someone not have heard of carbon taxes or
| balanced growth paths?
|
| What's funnier is that you both are revealing that either you
| don't know the methodology of solving economics models, or
| don't know what the physics equations represent. Nature acts to
| minimize convex functions while economic agents move to
| maximize concave ones. Of course the solutions end up the same.
|
| Don't think that our utility functions are correct or actual
| people act differently? Honestly only having profits to
| maximize is just a good first order approximation. That's why
| we have an entire field working on supposedly irrational
| behavior and backing up everything with data and experiments.
|
| Now economics _does_ have its huge problems, but things aren 't
| as simple as this. But if you still see a huge problem, just
| get your correct model, back your results with data and enjoy
| getting the Nobel prize.
| choward wrote:
| > But if you still see a huge problem, just get your correct
| model, back your results with data and enjoy getting the
| Nobel prize.
|
| The "Nobel prize" for economics isn't a real Nobel prize. It
| was made up to make economists feel better.
|
| From Wikipedia:
|
| > He explained that "Nobel despised people who cared more
| about profits than society's well-being", saying that "There
| is nothing to indicate that he would have wanted such a
| prize", and that the association with the Nobel prizes is "a
| PR coup by economists to improve their reputation"
|
| https://en.m.wikipedia.org/wiki/Nobel_Memorial_Prize_in_Econ.
| ..
| fhood wrote:
| Seconded. Blindly implying that economists of all groups are
| blatantly appropriating math from other fields without
| understanding it seems a step too far to me. I'm sure it
| happens, but I've worked in conjunction with people writing
| computer models to study certain economic subsystems and
| virtually everybody involved had a serious academic
| background in mathematics.
| munchler wrote:
| Based on other people in the field whose responses were quoted
| at the end of the article, I suspect this phenomenon may still
| be occurring. Specifically:
|
| "David Pincus, a psychologist at Chapman University who
| specializes in the application of chaos theory to psychology"
|
| "Stephen Guastello, a psychologist specializing in nonlinear
| dynamics at Marquette University"
| [deleted]
| j7ake wrote:
| This misuse and abuse of methods will continue as long as you can
| make a big splash in the field.
|
| The technical correction of the misuse afterwards can then be
| tacked on to the end of your article later, with absolutely no
| recourse or penalty to your career.
| unixhero wrote:
| What is middle aged of any relevance???
| fluorinerocket wrote:
| richardfey wrote:
| Here's the key passage (SPOILER ahead):
|
| > Losada had simply lifted the numbers that Lorenz used in 1963
| to explain his method in the abstract, numbers used merely for
| illustrative purposes. Losada's results, along with the pretty
| butterfly graphs Brown had been shown in class, were essentially
| meaningless.
| sampo wrote:
| > But in attempting to draw an equivalence between the physical
| flow of liquids and the emotional "flow" of human beings, Losada
| had simply lifted the numbers that Lorenz used in 1963 to explain
| his method in the abstract, numbers used merely for illustrative
| purposes. Losada's results, along with the pretty butterfly
| graphs Brown had been shown in class, were essentially
| meaningless.
|
| It was a bit crazy in the late 1990s. Chaos theory had been
| popularized, and the social sciences copied fashionable buzzwords
| and buzzconcepts from chaos theory, and used them totally out of
| context. Maybe it was crazy in the 1920s, too? Einstein's theory
| of relativity had become popular, and sciences that had nothing
| to do with gravitation, copied the "everything is relative"
| slogan.
| roywiggins wrote:
| > In a valid fluid dynamics problem, the numbers plugged into the
| equation must correlate to the properties of the fluid being
| studied. But in attempting to draw an equivalence between the
| physical flow of liquids and the emotional "flow" of human
| beings, Losada had simply lifted the numbers that Lorenz used in
| 1963 to explain his method in the abstract, numbers used merely
| for illustrative purposes. Losada's results, along with the
| pretty butterfly graphs Brown had been shown in class, were
| essentially meaningless.
|
| Incredible. So it wasn't just a matter of, say, applying an
| equation beyond its domain, or generating the inputs to the
| function in a sketchy way, or hacking the inputs to the function
| until they got an answer they liked, but _literally copying
| example parameters from a 1963 paper_.
|
| The "Complex Dynamics of Wishful Thinking" paper is very
| readable:
|
| https://physics.nyu.edu/faculty/sokal/complex_dynamics_final...
| FabHK wrote:
| It is very readable, and it utterly nihilates Losada's papers.
| It is hard to believe that anyone could have taken them
| seriously, let alone let them get published. Sigh. Nice touch
| that Sokal helped debunking it.
|
| I wonder to which extent this taints the whole of "Positive
| Psychology", which I was moderately excited about.
| dekhn wrote:
| Falsifying large parts of the extant literature is trivial.
| dang wrote:
| " _Please don 't post shallow dismissals, especially of other
| people's work. A good critical comment teaches us something._"
|
| https://news.ycombinator.com/newsguidelines.html
| dekhn wrote:
| https://blog.scienceexchange.com/2012/08/the-
| reproducibility...
|
| "In the last year, problems in reproducing academic research
| have drawn a lot of public attention, particularly in the
| context of translating research into medical advances. Recent
| studies indicate that up to 70% of research from academic
| labs cannot be reproduced, representing an enormous waste of
| money and effort," said Dr. Elizabeth Iorns, Science
| Exchange's co-founder and CEO. "In my experience as a
| researcher, I found that the problem lay primarily in the
| lack of incentives and opportunities for validation--the
| Reproducibility Initiative directly tackles these missing
| pieces."
|
| https://journals.plos.org/plosmedicine/article?id=10.1371/jo.
| .. "Simulations show that for most study designs and
| settings, it is more likely for a research claim to be false
| than true. Moreover, for many current scientific fields,
| claimed research findings may often be simply accurate
| measures of the prevailing bias."
|
| but I didn't really need to add that to make my comment any
| better.
| dang wrote:
| I think it's pretty shallow still, even with that, because
| it's just a generic dismissal. Most readers are already
| familiar enough with this phenomenon that your comment
| amounts to a knee-jerk reaction--literally the most obvious
| thing one might say in the closest generic category. That's
| a marker of a bad HN comment, because it points to
| shallower, more generic discussion.
|
| Good comments are reflective, not reflexive: https://hn.alg
| olia.com/?dateRange=all&page=0&prefix=true&sor....
|
| The more generic a discussion gets, the less interesting it
| is: https://hn.algolia.com/?dateRange=all&page=0&prefix=tru
| e&que...
| oneepic wrote:
| How can you call the guy a CS major? He was a grad student in
| psychology, right?
| jvanderbot wrote:
| "The married father of two had graduated from Cambridge
| University in 1981 with a degree in computer science, and spent
| most of his career as an IT networks operator at an
| international organization in Strasbourg, France."
|
| Pretty early in the article.
| ineedasername wrote:
| Yes, but the context was also with that person as a current
| student, which was not in CS, and his CS background didn't
| have much bearing on what he discovered. It would have been
| just as relevant to say _" Father of two debunked..."_
|
| A better title would have been "New Student in APS debunks
| core finding in the field."
| jvanderbot wrote:
| HN readers are predominately CS/IT folks, and would
| probably relate better to that aspect of his person than
| others. That's certainly why _I_ read the article.
|
| Should I emphasize that he lives in the UK?
|
| "UK citizen debunks positive psychology"
|
| Or should I emphasize the statehood of the person debunked?
|
| "UK citizen debunks US professor with one neat trick"
|
| Just trying to summarize the relevant parts of the story
| without clickbait -- which the article did pretty well I
| thought.
| PaulDavisThe1st wrote:
| Going the full clickbait: "You'll never believe how this
| aging British empty-nested IT guy took down a US
| professor with one neat trick!"
| ineedasername wrote:
| The relevant part was his status as a APP student, so
| that's what I would go with for the headline.
|
| Using an irrelevant bit _because_ it will get more clicks
| seems to fall under the "click bait" domain, especially
| when he was a grad from CS, not a current major.
|
| Personally I would be _more_ interested to know that it
| was a newbie student to an academic field debunked a core
| tenet. "New APP Students Debunks Core APP Finding" is
| much more interesting to me than a coincidence of the
| former academics pursuits with mine. It implies the
| question, "If a brand new student to the field saw this
| error so quickly, why didn't anyone else?"
| kwhitefoot wrote:
| But that wouldn't attract so many HNers to comment.
| mywittyname wrote:
| Am I the only one who hates the phrase, " __ major"? One can
| "major" in a subject for all of three weeks. It's not some
| indicator of any sort of knowledge. At least the phrase, "___
| graduate" implies that they at least learned enough to be
| issued a degree.
| Jeff_Brown wrote:
| jvanderbot wrote:
| He traced the math behind it back to a computational fluid
| dynamics paper, which the debunkees had used equations from
| with no context. He cold emailed professors asking for feedback
| of his organized findings, got back a 3000 word article based
| on it, and with help from a mathematician and psychologist in
| smoothing the writeup, and ultimately got it published in a top
| journal and the offending results recognized as flawed. The
| general crusade continues.
|
| The result in question was the legendary 3-1 positive to
| negative tipping point between flourishing individuals and
| languishing ones.
| PaulDavisThe1st wrote:
| Brown did not write the "3000 word critique". From TFA:
|
| > Sokal, "came back with my draft one day," Brown says, "and
| had written 3,000 words."
| jvanderbot wrote:
| Ah! My reading comprehension strikes again. Edited, thank
| you.
| at-fates-hands wrote:
| _And through it all, the three met only once in person, when
| Friedman was in London to give an academic address. It was a day
| spent scribbling equations in chalk on blackboards and talking
| strategy - a debunkers' bonhomie._
|
| I truly miss this about academics. Where you get a chance to sit
| down and think freely and pick someone else's brain in the
| pursuit of some hypothesis you're chasing down.
| setgree wrote:
| I worked with Nick a few times in a previous job and he was a
| reviewer on a paper I wrote; he was really nice and professional,
| and his feedback was constructive and incisive.
|
| The movement he's part of is fundamentally disagreeable (in the
| 'big five' sense [0]), and some of its prominent characters, as
| you might expect, have reputations for being interpersonally
| difficult [1]. Collectively they've been labeled "data thugs" [2]
| and accused of "methodological terrorism" [3], which is why I
| think Nick's fundamental goodness is an especially valuable
| asset. We'd gain a ton of social value from certain reforms to
| the scientific research apparatus, and, IMO, being kind when
| delivering pointed critiques helps a difficult pill go down.
| That's different than pulling your punches; it just means staying
| focused on the thing you actually want to change.
|
| [0] https://en.wikipedia.org/wiki/Big_Five_personality_traits
|
| [1] https://medium.com/@OmnesRes/the-dumbest-fucking-thing-
| ive-e...
|
| [2] https://www.science.org/content/article/meet-data-thugs-
| out-...
|
| [3] https://statmodeling.stat.columbia.edu/2016/09/21/what-
| has-h...
| tomrod wrote:
| I wanted to hate [1], but find myself agreeing quite a lot on
| Jordan's substance. Well done to the writer!
|
| What is this about data thugs and posses?
| [deleted]
| fhood wrote:
| Hahaha, it is a bit of a catch-22 though isn't it. In debunking
| a paper like this you have to understand that your are cutting
| away the foundation that people built their careers on, and you
| may be fundamentally damaging their academic trajectory.
| Doesn't mean you shouldn't do it, just that being a bit of an
| asshole probably helps.
| motohagiography wrote:
| If this generalizes, seems like someone with a graph database,
| some basic calculus, and an archive of papers citation data
| could pull the rug out from under entire disciplines,
| specifically ones that are informing public policy these days.
| These "data thugs," they seem like what we hoped hackers would
| be.
| native_samples wrote:
| They have done but public policy doesn't respond.
|
| For example one of the "data thugs" (James Heathers) wrote a
| tool that could detect impossible numbers in psych papers,
| like means/std devs that couldn't possibly have been the
| result of any allowable combination of results. Some very
| high percentage of papers failed, I think it was 40%.
|
| And of course psychology isn't the worst. Epidemiology is
| worse than psychology. The methodological problems there are
| terrifying. Good luck getting public policy to even accept
| that it's happened, let alone do anything about it.
| lmc wrote:
| Not that simple, and that was something missed in the
| original article - not all citations are equal. You have to
| know the context of each. Like, it could've been 1 positive
| citation, and 349 calling it bullshit... or vice versa.
| motohagiography wrote:
| Indeed, the beauty of using a graph is you can see the
| direct citations, and then secondary and teritiery ones and
| then pull apart the clusters to see if there is any
| opportunity to pull on mathematical threads.
|
| A query like, given paper X, show papers that cite it,
| papers that cite those, and then ones that cite both.
| Cycles in the graph might show citation rings as well (as I
| think has happened in the past), but this would be a
| challenge for young people to apply their high school
| calculus skills to verifying claims in social science
| research.
|
| A breadth first search across papers using citations
| selects which paths you want to do your depth first
| validation on. We do the same thing in vulnerability
| research with software dependencies, and picking a bad-math
| vuln in a discipline and tracing its impact though, or
| picking clusters of tightly dependent research and finding
| vulns in them seems like an opportunity for an enterprising
| outsider who never wants a job from any of these people.
| whimsicalism wrote:
| I'm unsure what your second citation is for, they don't use the
| word "data thugs" but the author certainly seems annoying
| (although likely correct about the points they're making as far
| as I read).
|
| e: the parent comment has now been edited for clarity, keeping
| my comment for posterity
| [deleted]
| setgree wrote:
| The piece is written by a member of the posse [0] who can be,
| as I think the above quoted piece demonstrates, somewhat
| abrasive
|
| [0] https://medium.com/@OmnesRes/the-circle-of-data-thug-
| life-81...
| MegaButts wrote:
| If that's considered abrasive, we've really lost out on
| actual communication for the sake of bullshit etiquette.
| aidenn0 wrote:
| It is hard for me to comprehend a point of view where
| that article is not considered abrasive in tone. Whether
| or not it bothers you is another question.
|
| The tile is in all caps "THE DUMBEST FUCKING THING I'VE
| EVER READ" and challenges someone to a cage fight at the
| end. The entire rest of the article could have been
| honeyed sweetness (which clearly it's not), and that
| could be enough to call it abrasive.
|
| One person I showed it responded very favorably,
| commenting that the author clearly considered it
| "...important to get the basic points across violently
| vecause no one is listening to the non-violent
| arguments." i.e. this is abrasive, but abrasive to a
| purpose.
| whimsicalism wrote:
| > If that's considered abrasive, we've really lost out on
| actual communication for the sake of bullshit etiquette.
|
| Did we read the same article? I'm confident that we can
| have actual communication that is both less obnoxious and
| not hamstrung by "bullshit etiquette."
| [deleted]
| DantesKite wrote:
| What's funny is the whole field of Big Five is fundamentally a
| fraud because it has both correlation and dimensionality
| problems.
|
| And using it to describe Nick, who apparently took down another
| positive psychology finding, is ironically funny.
| HelloMcFly wrote:
| > the whole field of Big Five is fundamentally a fraud
|
| Do you have more reading? The Big 5 is hardly perfect and
| merits critique, but to call it "fundamentally a fraud"
| frankly feels like the conclusion of a "topic tourist" that
| read 1-2 critiques and made up their mind. It's not
| theoretically-derived, it relies on factor analysis, its
| utility does not all cultures, the five traits are not fully
| orthogonal. I'm not here to champion it. But comparing Big 5
| to Positivity Ratios or Power Posing seems hyperbolic at
| best, particularly given its usage in situations where
| validity must be demonstrated.
|
| But I've changed my mind on a lot of things. I'm prepared to
| change my mind on this if you've seen or read something I
| haven't.
| JackFr wrote:
| Consider the whole type A personality thing - it's pure
| nonsense from a scientific perspective (created by tobacco
| companies to explain why certain people [smokers] had more
| heart attacks.)
|
| But at this point it's still a useful cultural shorthand to
| describe certain characteristics we subjectively experience
| in others.
| HelloMcFly wrote:
| Yes, but that's a) not the Big 5, and 2) the worst
| example of personality tests/models where you're put into
| a _category_ (which will show very low reliability over
| time, even if your responses show some).
| runarberg wrote:
| It's been a while since I was studying this topic but if I
| remember correctly, there is fundamental problem of a lack
| of justification for operational definition in the whole
| field of psychometrics. This applies to most (all)
| personality tests including the big 5.
|
| Because the whole theory is based around cluster analysis
| it is pretty important that you can justify the data that
| is going into the analysis if you want it to be reveling
| any truth outside of the model, otherwise you end up with
| what is called _"junk-in-junk-out"_. As far as I know, this
| justification is still lacking 40 years after this theory
| surfaced.
|
| I think that the big 5--and personality psychology in
| general--might not have the same glaring issues as positive
| psychology. But solid science it is not. Fraud might be
| overreaching, but I would definitely categorize it as
| pseudoscience.
| derbOac wrote:
| Lack of justification for operational definition?
|
| That's pretty subjective and could be leveled against
| almost anything behavioral or psychological.
|
| The thing about the Big Five is that it has surfaced in
| all sorts of contexts. You can maybe say maybe the 5 per
| se is not well justified (as opposed to 4 or 6 or 7, for
| example), but if you take enough ratings of a person,
| some variant of those 5 will _probably_ work as
| reasonable summaries of the ratings, and they will
| account for a substantial chunk of their predictive
| variance. The thing too, is that if you take other types
| of variables, like clinical symptom ratings, or
| diagnoses, you start to see roughly similar types of
| attributes become prominent.
|
| The Big Five is a descriptive model of how people
| perceive others. There's a lot of evidence for certain
| mechanistic processes being heavily involved in some
| (e.g., positive emotion in extraversion, negative emotion
| in neuroticism, behavioral control with
| conscientiousness, etc.) but I'm not sure the original
| idea behind the Big Five was mechanistic -- it was a
| hypothesis about major dimensions that could summarize
| social perceptual data. It's like classification in
| biology pre-DNA era. People have some ideas of how things
| go together, and find it useful for organizing
| descriptions and measuremnts.
|
| It's like if you did unsupervised DL modeling of all the
| videos involving humans you can find on the web, and
| found that their classification could be accounted for by
| 5 major vectors, almost all the time, regardless of
| sampling. Wouldn't you want to know that?
|
| Many other measures are very mechanistically well-
| justified but lack ecological validity, in the sense that
| they are very narrow predictively and not well outside of
| laboratory contexts. That's fine, there's a tension there
| between predictive bandwidth and depth, but if you want
| any kind of rating of a human being's behavior and
| experience, you enter at your own risk if you think
| you'll measure something that's radically different from
| the Big Five (or something subsumed by cognitive
| measures). Can you do it? Sure, but a lot of the time no
| (see: grit).
| runarberg wrote:
| What constitutes as an adequate justification for use of
| an operational definition within a model is subjective
| indeed. However there is usually a point in the life of a
| theory where the gathered evidence are sufficient enough
| that a scientific consensus starts to form that the
| operational definition is justified. I'm not aware that
| that has happened in the 40-odd year history of the Big 5
| personality theory.
|
| The 5 personality traits may be overarching within the
| field of psychometrics and they may indeed be useful to
| describe behavior, however you still need to justify that
| said behavior is not easier described using different
| models, and this is where personality psychologists
| usually fails in justifying their operands.
|
| Works criticizing the model range from using totally
| different constructs (such as priming, positive
| reinforcement, universal grammar, brain dopamine level
| socio-economic status etc.)--which don't rely on
| psychometrics at all--to claiming that the behavior
| psychometricians are predicting are actually not that
| useful (e.g. predicting 'high confidence' is not that
| useful if 'high confidence' does not result in a
| significant behavior which isn't better predicted without
| made up operands).
|
| If you were an early astronomer and you constructed the
| notion of 'epicycles' to simplify your model of planetary
| motion. You may use these 'epicycles' to justify your
| prediction, however you may not use a successful
| prediction to justify the existence of epicycles. Your
| epicycles may be useful until someone comes along and
| deems them unnecessary since planetary motion is better
| described by using elliptical orbits.
|
| Of course this could go the other way as is the case with
| particle physics and the atom. However given the amount
| of research, success of rival theories, the failure of
| psychometricians from making useful predictions outside
| of their narrow field that isn't better explained with
| alternative theories, I have high doubts that the Big-5
| personality traits (and any theory of personality using
| psychometrics for that matter) is anything but
| pseudoscience.
| HelloMcFly wrote:
| I appreciate your response, but think I'd have to read
| more to be nearly as convinced as you. I'm familiar with
| the critiques you mention, but your conclusion as to the
| model's merits go beyond what I've seen other critiques
| make. As a model grounded in theory it is lacking, but as
| an explanatory construct to predict patterns of behavior
| and outcomes in specific settings (e.g., knowledge work)
| it persists for a reason. I've peer reviewed papers
| criticizing it, but none "debunking" it.
|
| Personally my biggest gripe personally is that it is
| represented as a model of total personality, but that's
| definitely not true. It's just just representing a larger
| "personality space" than most other constructs. At least
| the outcome isn't placing one into a discrete category.
| runarberg wrote:
| When I was doing psychology over a decade ago personality
| psychology was actually my biggest gripe. The way that I
| saw it, was that it wasn't explaining anything which
| didn't have better explanation using a different theory.
|
| Now a decade later--being a little more class conscious--
| I can actually see how this is problematic. When a better
| explanation can come from sociology and has to do with
| class and economic status, measuring people based on a
| theory that lacks justification to justify one hire over
| another is problematic. When you ascribe it was because
| of an operationally defined concept (read: made up; based
| on data analysis) and name it _personality_ that seems
| like a lame excuse to hire from your ingroup and
| excluding your outgroup.
|
| These tests are robust, I know that, however robustness
| alone is not enough to justify a theory. These tools
| might be useful (or they may be dangerous) but while
| there is no justification for the operationally defined
| terms it cannot be used to justify terms outside of the
| model. That is pseudoscience. You can only to explain
| things inside of the model, and as such it is pretty
| limited as a theory.
| Double_Org wrote:
| Note that "operationalization" implies a fairly specific
| set of epistimolgical and ontological approaches which do
| not necessarily require that what is being measured has a
| one-to-one correspondence to a 'real' entity.
| runarberg wrote:
| Indeed. You can operationally define anything you want
| within your model. If done carefully, a good operational
| definition may simplify your model quite a bit. (A bad
| operational definition, on the other hand, will almost
| certainly make your model overly complex and can be quite
| detrimental).
|
| When you use your model to infer about a real world
| phenomena you have to be careful how you treat your
| operational definition. If you use it to make prediction,
| you cannot make a claim that what your operand _caused
| it_ , not until you go into the real world and find it.
| If your model is successful you may use your operand to
| describe your prediction, but you have to justify why
| your operand is necessary, a better model may exist which
| doesn't use an operand at all.
|
| A successful model is neither a sufficient nor necessary
| condition for proving an operand exists.
| bumby wrote:
| Can you share more or point to further reading? I originally
| came across the Big Five because a psychologist was using it
| to debunk the idea of "grit" as being a predictor of success.
| (The claim was grit was not a new concept and essentially
| just a repackaging of the conscientiousness Big Five trait.)
|
| It would be interesting (and another example of irony) if the
| Big Five itself was debunked.
| efsavage wrote:
| IANA data scientist, but I would hope that 4 decimal places in
| the context of psychology would trigger anyone's BS meter.
| anyfactor wrote:
| I think there is inverse relation of how frequently a number is
| written everywhere with it's degree of decimal places. It is a
| convenience thing. Also if the number is generated on a
| computer and used there subsequently 4 decimal is plenty for
| social science and business research/purpose IMO.
| klabb3 wrote:
| I think parent means the opposite, that 4 decimal places is
| really high precision in a field with lots of measurement
| bias and noise.
| anyfactor wrote:
| TIL. This is something new I learned. Can you tell me what
| to google to learn about this bias?
| waterheater wrote:
| https://en.wikipedia.org/wiki/Statistical_significance
|
| When the Higgs boson was declared "discovered," the
| particle was found to exist with a five-sigma certainty.
| This brief article does a great job of explaining what
| that means: https://www.r-bloggers.com/2012/07/the-higgs-
| boson-5-sigma-a...
|
| Physicists got to that point (very roughly) by
| constructing complex mathematical models and building
| large machines to test those models.
|
| In contrast, leading psychology models change with the
| seasons, and large-scale human experiments cannot be
| (ethically) conducted with the control as seen in
| physics. Because psychology experiments have substantial
| environmental variability, drawing robust conclusions in
| psychology is exceedingly difficult.
|
| This fact should mean that very-high-certainty findings
| in psychology studies must be subjected to the highest
| scrutiny.
| CRConrad wrote:
| I.e, far from being "2.9013", the question is whether
| this (bogus?) factor should be rounded to 3, or perhaps
| better "between 1 and 10".
| [deleted]
| okl wrote:
| I think they should have sent in their paper somewhere other than
| APA. Don't expect courtesy in consequential confrontations. I
| think similar about responsible disclosure sometimes. If a
| company played dirty in the past why bother with accomodating
| them.
| duxup wrote:
| I think they wanted to give the APA a chance to make the right
| call / improve, and it also seems they found the peer process
| valuable and they wanted to get that feedback.
|
| Seems like it worked out.
| CRConrad wrote:
| > Seems like it worked out.
|
| Yeah, kind of... Mainly by exposing _how reluctant_ the APA
| was to make the right call / improve.
___________________________________________________________________
(page generated 2022-02-07 23:01 UTC)