[HN Gopher] Berkson's Paradox
___________________________________________________________________
Berkson's Paradox
Author : luu
Score : 70 points
Date : 2022-11-20 03:10 UTC (19 hours ago)
(HTM) web link (en.wikipedia.org)
(TXT) w3m dump (en.wikipedia.org)
| mjburgess wrote:
| The whole family of statistical paradoxes like these arise, imv
| (and perhaps that of Pearl's) due to a failure to distinguish
| between statistical and explanatory (ie., causal) analysis.
|
| The whole machinery of statistics is insensitive to whether X
| causes Y, Y causes X, neither cause each other, or there is an
| intermediate cause.
|
| To see this, take any graph with any statistical model, and flip
| the axes. The analysis is the same, but the direction of
| causation implicitly reversed. Indeed, take any of these
| paradoxes (esp. simpson's).
|
| Nevertheless for much of the recent history of applied statistics
| these distinctions have not been made (esp. because it has been
| developed within the experimental contexts of dubious fields).
|
| It is important to state however, that no technique or quantity
| of statistics is ever sufficient to credit any association with
| being an explanation. To credit any association with any
| scientific theory, and hence neither with any properties.
|
| No quantity of associative statistical research into IQ thereby
| implies such a thing exists (cf. the reification fallacy).
|
| To transcend statistics into science one needs experiments, those
| which can control variables, and hence provide a non-statistical
| interpretive framework which demonstrates these variables are
| explanatory, and hence gives credence to the theories which
| explain them.
| perfecthjrjth wrote:
| This. Very interesting take, indeed.
| rahimnathwani wrote:
| Related:
|
| * https://twitter.com/page_eco/status/1373266475230789633
| (discussion: https://news.ycombinator.com/item?id=26566810)
|
| * https://erikbern.com/2020/01/13/how-to-hire-smarter-than-the...
| ouid wrote:
| for whom is this unintuitive?
|
| pick two random numbers whose sum is 0.
| nerdponx wrote:
| > pick two random numbers whose sum is 0.
|
| It's unintuitive if you don't realize that the numbers sum to
| 0. This stuff can creep up on you in real-world data without
| realizing it.
| bmacho wrote:
| Is that still happening? I'd guess that statisticians figured
| it out by now? Is it hard, or impossible maybe?
|
| But I agree with GP, how is this surprising? The example in
| the article is stupid, and I am not convinced whether there
| are any real surprising or counter-intuitive examples.
| mxwsn wrote:
| This can happen all the time in data science. Just add a
| few layers of bureaucracy and department separations
| between data gathering, cleaning/filtering, and analysis
| and it's easy to make the wrong conclusions on observed
| correlations without carefully thinking through all the
| filtering steps that might induce Berkson's paradox.
| ZephyrBlu wrote:
| It's possible to implicitly filter your population, like in
| the hospital example given on the wiki page.
|
| Any situation where you make assumptions about the
| population could cause Berkson's paradox.
|
| The Ellenberg example on the wiki page is a classic
| situation. Less attractive people are filtered out of the
| dating pool due to standards. This is reasonable, but it
| creates Berkson's paradox.
|
| It's often hard to realize this is happening.
| dang wrote:
| Recent and related:
|
| _Brilliant jerks, crazy hotties, and other artifacts of range
| restriction (2019)_ -
| https://news.ycombinator.com/item?id=33676648 - Nov 2022 (73
| comments)
|
| Also:
|
| _Berkson 's Paradox_ -
| https://news.ycombinator.com/item?id=26566810 - March 2021 (39
| comments)
|
| _Berkson 's Paradox_ -
| https://news.ycombinator.com/item?id=18667423 - Dec 2018 (21
| comments)
|
| _Berkson 's Paradox_ -
| https://news.ycombinator.com/item?id=8264252 - Sept 2014 (20
| comments)
___________________________________________________________________
(page generated 2022-11-20 23:00 UTC)