[HN Gopher] Dumb statistical models, always making people look bad
___________________________________________________________________
Dumb statistical models, always making people look bad
Author : hackandthink
Score : 58 points
Date : 2025-04-19 13:12 UTC (2 days ago)
(HTM) web link (statmodeling.stat.columbia.edu)
(TXT) w3m dump (statmodeling.stat.columbia.edu)
| delichon wrote:
| > why it's often hard to demonstrate the value of human knowledge
| once you have a decent statistical model.
|
| This seems to be a near restatement of the bitter lesson. It's
| not just that large enough statistical models outperform
| algorithms built from human expertise, they also outperform human
| expertise directly.
| gopalv wrote:
| > they also outperform human expertise directly
|
| When measured statistically.
|
| This is not a takedown of that statement, but the reason we've
| trouble with this idea is that it works in the lab and not
| always in real life.
|
| To set up a clean experiment, you have define what success
| looks like before you conduct the experiment - that the output
| variable is defined.
|
| Once you know what to measure ahead of time to determine
| success, then statistical models tend to not be as random as a
| group of humans in achieving that target.
|
| The variance is bad in an experiment, but variance jitter is
| needed in an ever changing world even if most variants are
| worse off.
|
| For example, if you can predict someone's earning potential
| from their birth zipcode, it is not wrong and often more right
| than otherwise.
|
| And then if you base student loans and business loan interest
| rates on the basis of birth zipcodes, the original prediction
| does become more right.
|
| The experimental version that's a win, but in real life that's
| a terrible loss to society.
| bobsomers wrote:
| > > they also outperform human expertise directly
|
| > When measured statistically.
|
| THANK YOU. It's mildly infuriating how often people forget
| that one of the things most human experts are good at is
| knowing _when_ they are looking at something that is likely
| in distribution vs. out of distribution (and thus, updating
| their priors).
| nitwit005 wrote:
| You don't even need a statistical model. We make checklists
| because we know we'll fail to remember to check things.
|
| Humans are tool users. If you make a statistical table to consult
| for some medical issue, you've using a tool.
| taeric wrote:
| I was going to say that it doesn't have to be a statistical
| model. Notable that statistical models are already seen as less
| than complete analytical models, for many people. (I think that
| is almost certainly a poor way of wording it? Largely just
| trying to say that F=ma and such are also models that don't
| have conditional answers.)
|
| At any rate, I'm curious on some of the readings this post
| brings up. I'm also vaguely remembering that human's can have
| some odd behaviors where requiring justification or reasoning
| of decisions can sometimes provide more predictable decisions;
| but at a cost that you may not fully explore viable decisions.
| dominicq wrote:
| As a matter of practicality, it seems that you professionally now
| want to be firmly in the tails of the data distribution for your
| field, e.g. expert in those things that happen rarely.
|
| Or maybe even be in a domain which, for whatever reason, is
| poorly represented by a statistical model, something where data
| points are hard to get.
| rawgabbit wrote:
| OTOH. The blog mentions that humans excel at novel situations.
| Such as when there is little training data, when envisioning
| alternate outcomes, or when recognizing the data is wrong.
|
| The most recent example I can think of is "Frank". In 2021,
| JPMorgan Chase acquired Frank, a startup founded by Charlie
| Javice, for $175 million. Frank claimed to simplify the FAFSA
| process for students. Javice asserted the platform had over 4
| million users, but in reality, it had fewer than 300,000. To
| support her claim, she allegedly hired a data science professor
| to generate synthetic data, creating fake user profiles. JPMorgan
| later discovered the discrepancy when a marketing campaign
| revealed a high rate of undeliverable emails. In March 2025,
| Javice was convicted of defrauding JPMorgan.
|
| IMO an data expert could have recognized the fake user profiles
| through the fact he has seen e.g., how messy real data is, know
| the demographics of would be users of a service like Frank
| (wealthy, time stressed families), know tell tale signs of fake
| data (clusters of data that follow obvious "first principles").
| 3abiton wrote:
| It's unfortunate how under appreciated is statistics, in nearly
| all (spare academic) positions that I occupied, mostly in the
| technical domain interacting with non-technical stakeholders,
| anectodal evidence always take priority compared to statistical
| backed data, for decision making. It's absurd sometimes.
| bsder wrote:
| This is because the _correct_ answer is rarely the _politically
| palatable_ answer.
___________________________________________________________________
(page generated 2025-04-21 23:00 UTC)