[HN Gopher] The t-test was invented at the Guinness brewery
       ___________________________________________________________________
        
       The t-test was invented at the Guinness brewery
        
       Author : rmason
       Score  : 370 points
       Date   : 2024-05-26 20:55 UTC (1 days ago)
        
 (HTM) web link (www.scientificamerican.com)
 (TXT) w3m dump (www.scientificamerican.com)
        
       | mmastrac wrote:
       | Interesting read. I don't think this came up in my stats classes:
       | 
       | > Gosset solved many problems at the brewery with his new
       | technique. The self-taught statistician published his t-test
       | under the pseudonym "Student" because Guinness didn't want to tip
       | off competitors to its research. Although Gosset pioneered
       | industrial quality control and contributed loads of other ideas
       | to quantitative research, most textbooks still call his great
       | achievement the "Student's t-test."
        
         | jbjbjbjb wrote:
         | I always thought that name was strange but I never thought to
         | look it up. Stats books are so dry, they don't have the
         | inclination to share these kinds of stories.
        
           | TillE wrote:
           | The history of just about anything is very interesting, but
           | it's generally not relevant in a textbook which has a
           | specific purpose.
        
             | mp05 wrote:
             | I agree with the sentiment, but I always have wondered what
             | t-test a real engineer uses and why they only teach the
             | "Student" version. Given the context, a bit of a clarifier
             | would have been appreciated.
        
           | alpple wrote:
           | I wonder if dry reading means written without the influence
           | of drink. I couldn't find an answer online. But, if so, it
           | would be ironic to describe a stat book that ignored a brewer
           | as dry.
        
           | ayhanfuat wrote:
           | You probably missed it. This is something stat book authors
           | love to mention. I don't remember a stat intro book that
           | doesn't have a footnote for "Student t".
        
             | BeetleB wrote:
             | Just went through mine a few months ago. It definitely
             | doesn't have it.
        
               | whimsicalism wrote:
               | what book? i'm honestly curious because of how frequently
               | this story is repeated in stats textbooks
        
               | BeetleB wrote:
               | The author is Jay Devore.
        
           | nxobject wrote:
           | Wait until you hear about the bad blood between Fisher and
           | Pearson.
        
           | jldugger wrote:
           | > Stats books are so dry, they don't have the inclination to
           | share these kinds of stories.
           | 
           | It doesn't have to be that way. My pandemic lockdown read was
           | a 10 dollar Stats textbook[1], that comes with tons of
           | classic examples: the Salk polio vaccine, a prosecutor
           | misusing the multiplication rule using purely circumstantial
           | evidence ("what are the odds that police pulled over the
           | wrong couple matching 10 different pieces of description by
           | the victim?"), the classic Gallup poll showing FDR would
           | defeat Landon (versus incumbent _Literary Digest_ showing a
           | Landon win), Gosset's history with Guiness, the early history
           | of probability as gambling strategy, a controversy over
           | Mendel's data on pea plant heredity being _too_ clean, and so
           | on.
           | 
           | Sadly, while this book left me well prepared to apply
           | statistical reasoning in my day job, it's departure from
           | typical pedagogy left me feeling unprepared for further
           | reading based on perusal of Stats Wikipedia -- what's a
           | kernel? what's a moment? etc.
           | 
           | [1]: https://www.amazon.com/Statistics-Fourth-David-Freedman-
           | eboo...
        
             | madcaptenor wrote:
             | In a former life I taught some intro stat courses from this
             | book. It's a good book for an intro course for people who
             | aren't going on to further stat classes, although I think
             | for a current class I'd want something that acknowledges
             | how statistics and computers have gotten all tied up with
             | each other. (I don't have any recommendations - it's not my
             | job to know this any more.)
        
           | whimsicalism wrote:
           | sorry but this is a classic story mentioned in almost every
           | basic stats textbook i've read
        
           | roenxi wrote:
           | Doubly unfortunate because the philosophical aspects of
           | statistics are more important to students than most of maths.
           | There are something like 4 different schools of thought [0]
           | and people will have a natural propensity to one of them.
           | 
           | Although they all agree on the formulas and rigorous aspects,
           | it is actually a challenging proposition to comprehend what
           | someone is doing if you strongly see the world from one
           | perspective and don't realise that academics are potentially
           | approaching the interpretation in one of 3 other ways.
           | 
           | It adds a lot of dryness to the textbook because the author
           | can really only talk about the objective parts in an
           | introductory classroom setting. But if you're getting taught
           | by a frequentist and have a subjectivity bent it is easy to
           | spend a year or two confused before someone clues you in that
           | there are unresolved questions of interpretation.
           | 
           | [0]
           | https://en.wikipedia.org/wiki/Interpretation_of_probability
        
             | enasterosophes wrote:
             | > people will have a natural propensity to one of them.
             | 
             | I see what you did there
        
             | riffraff wrote:
             | Interesting, my university book of the subject was pretty
             | tiny but it did talk of the different interpretations, but
             | it only mentioned frequentist and bayesian. I did not
             | suspect the story was much more complicated.
        
           | orhmeh09 wrote:
           | I liked this one a lot
           | 
           | Abelson, R. P. (1995). Statistics as Principled Argument.
           | Psychology Press. https://www.routledge.com/Statistics-As-
           | Principled-Argument/...
           | 
           | > In this illuminating volume, Robert P. Abelson delves into
           | the too-often dismissed problems of interpreting quantitative
           | data and then presenting them in the context of a coherent
           | story about one's research. Unlike too many books on
           | statistics, this is a remarkably engaging read, filled with
           | fascinating real-life (and real-research) examples rather
           | than with recipes for analysis. It will be of true interest
           | and lasting value to beginning graduate students and seasoned
           | researchers alike. The focus of the book is that the purpose
           | of statistics is to organize a useful argument from
           | quantitative evidence, using a form of principled rhetoric.
           | Five criteria, described by the acronym MAGIC (magnitude,
           | articulation, generality, interestingness, and credibility)
           | are proposed as crucial features of a persuasive, principled
           | argument. Particular statistical methods are discussed, with
           | minimum use of formulas and heavy data sets. The ideas
           | throughout the book revolve around elementary probability
           | theory, t tests, and simple issues of research design. It is
           | therefore assumed that the reader has already had some access
           | to elementary statistics. Many examples are included to
           | explain the connection of statistics to substantive claims
           | about real phenomena.
        
         | mindcrime wrote:
         | At least they let him publish, albeit under a pseudonym. It
         | makes me wonder how many potentially useful discoveries were
         | made in industrial settings, and wound up being buried due to
         | management not wanting to risk leaking competitive information.
         | The good news, I suppose, would be if you believe that it's
         | rarely the case that only one person could ever discover
         | something. Then you can conclude that all (most?) such
         | discoveries were eventually (or will eventually be)
         | rediscovered independently.
         | 
         | On a related note... I wonder how much valuable research
         | disappears (more or less) when companies fold, get acquired,
         | etc. Take MCC[1] for example. I've been doing a lot of reading
         | lately that involves old papers from the 1990's on "agents" and
         | "multi-agent systems". And time and time again, in the
         | references, you'll see something like "MCC Technical Report
         | TR86-32791" or some-such. Occasionally said report can be found
         | online, but quite a few of them seem to be either hard - or
         | impossible - to find. Maybe there's an archive of physical
         | papers stored away somewhere, but FSM knows where the heck such
         | a thing would be, or how hard it would be to get access.
         | 
         | A similar situation came up a while back when we started
         | discussing "sharding" here on HN[2]. There was a lot of effort
         | spent trying to identify when the term first arose, and a lot
         | of evidence pointed to a particular paper that was internal to
         | CCA, who were acquired by Xerox. And now that original paper
         | seems to be unobtanium. The paper probably still _exists_
         | somewhere in the bowels of Xerox, but good luck ever getting
         | your hands on it.
         | 
         | [1]:
         | https://en.wikipedia.org/wiki/Microelectronics_and_Computer_...
         | 
         | [2]: https://news.ycombinator.com/item?id=36848605
        
           | schneems wrote:
           | > It makes me wonder how many potentially useful discoveries
           | were made in industrial settings, and wound up being buried
           | due to management not wanting to risk
           | 
           | Probably a lot. I've come to find out that some dinosaur
           | companies won't even let their programmers open up issues on
           | open source repos (forget sending patches or releasing their
           | own software).
           | 
           | The logic goes like this: if someone found the log4j zero day
           | before it was reported they could comb through all issues and
           | see the companies that the users worked for then try to
           | target them. In this case any comment would indicate possible
           | involvement.
           | 
           | The least bit of security, through the tiniest extra bit of
           | obscurity. Thankfully many of these companies are starting to
           | come around and realizing that a lack of involvement with
           | open source is more risky than accidental 3rd hand
           | information leaking (like what dependencies doesn't certain
           | company use).
        
             | vlovich123 wrote:
             | The easiest counter to this is that, to my knowledge at
             | least, it's easier to build a vulnerability scanner than to
             | scrape repos for more targeted attacks.
        
               | schneems wrote:
               | The "No lieutenant, your men are already dead" defense. I
               | like it.
               | 
               | I think that if your threat model includes nation states
               | (and the companies I was referencing above was largely
               | S&P500 financial institutions) then you have to think the
               | attacker also doesn't want to trip off any alarms with a
               | ham fisted port scan blasting the precious zeroday
               | exploit all over the internet. Your point is still
               | extremely valid though.
               | 
               | Which is why the counter I provided is that the best
               | defense is to get as many engineers' eyes on the problem
               | and in the codebase as possible to prevent or find it
               | before it becomes an issue. Things like lib XZ are scary,
               | but it's even scarier if not caught before it's in the
               | wild.
        
               | vlovich123 wrote:
               | The dirty secret is that nation states can get your
               | software dependency list pretty easily in a number of
               | ways (e.g. sending agents to meetups to nerd out & make
               | friends would be an expensive way but there's other
               | social engineering attacks I've observed).
               | 
               | The other secret is that monitoring software can't detect
               | anomalies ahead of time & the vulnerability scan will not
               | show up meaningfully any different than all the other
               | random traffic already happening. Your nation state can
               | hide it's vulnerability scan amongst all the other
               | vulnerability scanners already running (both legit as a
               | service when you request it against your server &
               | illegitimate actors trying to find a way in). So at best
               | a ham fisted search is unlikely to really tip your hand
               | in a meaningful way unless it requires having penetrated
               | a few layers of your security to begin with.
               | 
               | As for libxz, the scary part is that as an industry we
               | recognize the security challenge of not compensating
               | maintainers and yet we have lackluster responses to
               | fixing it (e.g. Google trying to pay OSS maintainers to
               | harden their security while completely ignoring that a
               | huge problem is that the maintainers can't devote full
               | time which opens an avenue for malicious actors to
               | overwhelm maintainers & take control socially as happened
               | with libxz).
        
           | xeonmc wrote:
           | relevant xkcd: https://xkcd.com/664/
        
             | mindcrime wrote:
             | Nice. I had not seen that particular XKCD before. Good one!
        
         | CrazyStat wrote:
         | This is such a great story, it should be included in every
         | intro stats class (I did, back when I taught intro stats).
         | 
         | Gosset didn't have the mathematical background to derive the
         | correct distribution theoretically, so he figured out what it
         | was by simulating drawing samples of different sizes thousands
         | of times and fitting curves. Simulating, in those days, meant
         | writing numbers on thousands of cards, then shuffling and
         | drawing a sample. Calculate the mean and standard deviation.
         | Repeat. Thousands of times. He published the result with an
         | apologetic shrug for not being able to prove it properly.
        
           | richrichie wrote:
           | along with compulsory Guinness tasting :)
        
           | ForOldHack wrote:
           | Um. Wow. That's quite a story. But, it's not real. "owever,
           | Guinness had a policy of not publishing company data, and
           | allowed Gosset to publish his observations on the strict
           | understanding that he did so anonymously."
           | 
           | I'm 1906, Gosset was the guest of Person at UCL, and since
           | Gosset had a First in Math, and Professor Pearson was the
           | leading mathematician and publisher of the Bell curve..
           | 
           | Gosset spent a year at UCL. University College London. A year
           | with an expert looking over his shoulder? I would think that
           | he would publish with an extreme amount of confidence,
           | forgoing the need for an apolocetic shrug, which I have never
           | ever heard of. Never, and I have a degree in math with a
           | minor in Statistics. They had playing cards. You are arguing
           | for large sample sizes, which is not economical - precisely
           | against the design of the test - which looks surprisingly
           | suspicious.
        
             | zinekeller wrote:
             | > But, it's not real. "owever, Guinness had a policy of not
             | publishing company data, and allowed Gosset to publish his
             | observations on the strict understanding that he did so
             | anonymously."
             | 
             | Except that this part _is_ true. Obviously, he is well-
             | known in academic circles, but Guinness did have a policy
             | against its employees to publish their research using a
             | pseudonym[1].
             | 
             | [1] Specifically, they can publish with three conditions:
             | 
             | 1) To not mention Guinness or its competitors,
             | 
             | 2) To not mention anything about beer (so topics
             | specifically about beer is forbidden), and
             | 
             | 3) To not publish using their surname (which in practical
             | effect is to publish using a pseudonym).
        
             | CrazyStat wrote:
             | > I would think that he would publish with an extreme
             | amount of confidence, forgoing the need for an apolocetic
             | shrug, which I have never ever heard of. Never, and I have
             | a degree in math with a minor in Statistics.
             | 
             | Good for you. As you might have guessed from reading that I
             | used to teach statistics, I have a _bit_ more than a minor
             | in the subject. Your attempt to appeal to authority, not to
             | put too fine a point on it, falls flat.
             | 
             | Just because you haven't heard of a thing don't mean it
             | isn't true. We can, after all, just read the original
             | paper:
             | 
             | > Before I had succeeded in solving my problem
             | analytically, I had endeavoured to do so empirically. The
             | material used was a correlation table containing the height
             | and left middle finger measurements of 3000 criminals, from
             | a paper by W. R. Macdonnell (Biometrika, i, p. 219). The
             | measurements were written out on 3000 pieces of cardboard,
             | which were then very thoroughly shuffled and drawn at
             | random. As each card was drawn its numbers were written
             | down in a book, which thus contains the measurements of
             | 3000 criminals in a random order. Finally, each consecutive
             | set of 4 was taken as a sample--750 in all--and the mean,
             | standard deviation, and correlation5 of each sample
             | determined. The difference between the mean of each sample
             | and the mean of the population was then divided by the
             | standard deviation of the sample, giving us the z of
             | Section III.
             | 
             | As for the apologetic shrug, in the course of the "analytic
             | solution" we have:
             | 
             | > The law of formation of these moment coefficients appears
             | to be a simple one, but I have not seen my way to a general
             | proof.
             | 
             | and then after a bit more math guessing the correct
             | distribution based on the moments
             | 
             | > Consequently a curve of Prof. Pearson's Type III may he
             | expected to fit the distribution of s2.
             | 
             | My story is slightly off; Gosset only used one sample size
             | rather than several different sample sizes. But he did use
             | simulation with thousands of hand written cards as his
             | approach to the problem, he did fail to prove the correct
             | distribution (moments are not sufficient to determine the
             | distribution), and he did publish with an apologetic shrug.
        
             | jll29 wrote:
             | Typo: guest of Person => Karl Pearson
        
           | 0xDEAFBEAD wrote:
           | It's interesting how mathematically shallow most stats
           | presentations are. In most other areas I've studied, you
           | start from some basics like axioms and gradually build up
           | machinery by proving theorems etc. But presentations I've
           | seen of the t-test focus on when and how to use it, without
           | going very deep into the derivation at all.
           | 
           | This leaves me skeptical of the movement to replace calculus
           | with stats in high school. It's true that an ordinary citizen
           | will find stats more useful. But for students who will go on
           | to become scientists and engineers, I think they should study
           | calculus. Calculus is a better on-ramp to the sort of rigor
           | you need in upper-level math. And I'm concerned that a bad
           | "cargo cult" stats class may be worse than no stats education
           | at all. Calculus education seems harder to screw up.
        
             | blt wrote:
             | Not sure why you are downvoted for this. Your points have
             | merit. I agree that stats classes can have a "cookbook"
             | flavor and do not generally lead to a deep understanding of
             | probability. But I would rather fix the stats classes than
             | abandon the topic.
             | 
             | Does anyone really argue to _replace_ calculus with stats?
             | I thought the idea was to offer both and let students
             | choose based on their interests.
        
               | jll29 wrote:
               | Propbabilities, combinatorics, logics and sets are the
               | most valuable things from high school maths that
               | benefitted me all the way through from teenager to
               | professor.
               | 
               | Calculus is intellectually stimulating, but for my line
               | of work (dealing with uncertaintly, risk, decision
               | making, AI), other parts of mathematics are more useful.
               | However, I would not argue calculus should be replaced. I
               | would argue for more "proper" maths to replace "recipe-
               | like" maths. It's more important to go deeper on a topic
               | than what the topic is.
        
               | CrazyStat wrote:
               | Combinatorics would make a great high school math course,
               | honestly. Lots of fun puzzles and very approachable.
        
             | CrazyStat wrote:
             | Calculus is also mathematically shallow in that sense: the
             | subject where you start with axioms and gradually build up
             | the machinery of calculus is (Real) Analysis, which is not
             | part of the standard calculus curriculum and which the vast
             | majority of people taking calculus will never study [1]. A
             | typical Calculus class expects students to memorize and use
             | things like trig function integrals which are presented
             | without proof; not so different from memorizing and using
             | statistical tests presented without proof, in my opinion.
             | 
             | In an intro statistics class I think conceptual depth is
             | more important than mathematical depth. It's more important
             | that students really understand the concept of
             | probabilistic inference, both hypothesis tests and
             | confidence intervals, than that they understand the
             | mathematical derivation of the t distribution [2].
             | 
             | Unfortunately intro stats classes often fail on this count
             | as well. One of the (many) straws that eventually broke my
             | desire to teach was a committee decision--a committee
             | composed entirely of people _not_ teaching intro stats--to
             | disallow students from bringing formula cheatsheets to
             | exams, effectively forcing us to make the students memorize
             | formulas rather than focusing on conceptual understanding.
             | 
             | [1] When I took Real Analysis there was a calculus class
             | that met right before us in the same room, which often ran
             | over so that the calculus students would be packing up as
             | we entered the room. One day as we're sitting down one of
             | them asks us what class we're there for, and then asks what
             | Real Analysis is all about, since he's never heard of it.
             | One of my classmates responded with the absolutely perfect
             | "Well, our homework last night was integrating x^2 from 0
             | to 1."
             | 
             | [2] I'd say the same goes for Calculus, for what it's
             | worth; actually understanding what an integral _means_ is
             | more important than being able to set up the Reimann sum
             | and take the limit.
        
         | petesoper wrote:
         | 54 years after I was mystified trying to parse the use of
         | "student" for this, here is the answer. Cool!
        
         | squirrel6 wrote:
         | This was in my textbook and my professor covered it as well!
         | Class of 17 here
        
         | wodenokoto wrote:
         | I always found "student" confusing in the name. Like, is there
         | a "professors t-test" or something?
         | 
         | I personally found a lot of peace after learning that tidbit.
        
         | jll29 wrote:
         | Many international conferences are regularly held in Dublin,
         | and attendees often visit the Guinness brewery as part of
         | conferences' social events, where a memorial plaque reminds
         | them of Gosset and his important contributions to statistics.
        
       | schneems wrote:
       | I did a talk in 2019 where I mention that tidbit. Of course, to
       | properly do it justice I had to bring a Guinness and open it on
       | stage.
       | 
       | Here's a video from EuRuKo that was filmed on a decommissioned
       | ocean liner converted into a hotel and conf space
       | https://youtu.be/Aczy01drwkg?si=lsVWAFv9f3eLc2fZ&t=1095
        
         | apsurd wrote:
         | damn, unfortunate the rabbit makes this unwatchable.
         | 
         | otherwise, i love beer and great story, i'll just read about
         | it.
        
           | schneems wrote:
           | I agree. You can read the talk here
           | https://www.schneems.com/2020/09/16/the-lifechanging-
           | magic-o....
           | 
           | I introduced the voiceover artist at the beginning, she's a
           | Japanese speaker that I found on fiver. I chose her because
           | the talk was being given at a Japanese speaking conference
           | (Ruby Kaigi).
           | 
           | I love the foil of having a second character on screen but
           | having an accented cartoon was not the right effect I was
           | going for.
           | 
           | I'm still experimenting with multiple characters in my talks
           | but my most recent one I did doesn't use any actors and I
           | read their lines like a narrator. I think the effect works
           | much better https://m.youtube.com/watch?v=-8UQMH6p-Mw&list=PL
           | 9oQ7yETvN12...
        
             | apsurd wrote:
             | ah, appreciate the reply, and the spirit of creativity.
             | 
             | Kinda feel like I got caught being overly critical and here
             | is the actual creator! Thanks for receiving it well and
             | also for putting your stuff out there.
        
               | schneems wrote:
               | I'm kinda glad you said something to be honest. I wanted
               | to mention the voiceover being cringe but it's also hard
               | to pre-apologize for something without raising a lot of
               | alarm bells.
               | 
               | I'm still proud of the overall talk. I'm glad I pushed a
               | limit, found it, and learned from it.
               | 
               | Your comment stated what you saw and how you felt about
               | it. I think you did a great job speaking up without
               | lashing out or talking down. I appreciate that.
        
           | whimsicalism wrote:
           | i would advise maybe toning down the strength of your
           | criticism when addressing the creator of the content /2c
        
             | schneems wrote:
             | I appreciate you sticking up for me. I can see this comment
             | being taken poorly by others. I also think we generally
             | need to learn to empathize that the creator might
             | experience our words different than how we mean them (and
             | therefore be kinder when we post).
             | 
             | In this case, I was aware that the rabbit is tough to watch
             | so I have the empathy for my viewer and the comment came
             | across as honest rather than harsh. I don't think you
             | should be downvoted. If it was a different time or place
             | then I 100% could have taken it the wrong way.
        
       | tombert wrote:
       | This doesn't surprise me; any industry involved with basically
       | any manufacturing seems like a perfect testing ground for
       | statistical methods. There's enough scale in these things where
       | subtle differences can save tons of money, so it can be
       | beneficial pretty quickly.
       | 
       | I always thought the t-test was clever just because of how simple
       | it was compared to more advanced stuff.
        
         | firesteelrain wrote:
         | We covered this in my Research Methods class as part of my
         | Systems Engineering Masters. Growing and agriculture in the
         | early 1900s benefitted from things like ANOVA, blocking
         | factors, nuisance factors, factors, levels, ranges and
         | experiment design such as full factorial design. It is taken
         | for granted these days. F statistic, F crit, etc
        
       | OisinMoran wrote:
       | Guinness was way ahead of their time and in many ways the Google
       | of their day (building accomodation, high pay, great perks). My
       | granddad worked there so all my dad's brother and sisters learned
       | to swim in the Guinness swimming pool!
       | 
       | Funnest tidbit is that the widget to get a good head from cans
       | won the best invention of the year, the year the internet wax
       | invented.
       | 
       | For an excellent piece on many more interesting bits about
       | Guinness, check this out: https://www.thefitzwilliam.com/p/no-
       | great-stagnation-in-guin...
        
         | mcphage wrote:
         | I don't think I'll ever be interested in Internet Wax.
        
         | rob74 wrote:
         | It also apparently pioneered the practice of companies getting
         | tax (or in their case, lease) concessions for "investing in
         | Ireland"...
        
         | wodenokoto wrote:
         | Since you called them the google of the day, it's only fitting
         | to mention that Guinness in "The Guinness book of record"
         | refers to the brewery.
         | 
         | And much like how we use Google to settle discussion in pubs
         | today, the book was published to do that very same thing, back
         | then.
        
       | chillingeffect wrote:
       | Wow, forcing visitors to individually disable 6 types of cookies.
       | :thumbsdown.tif:
        
       | jhbadger wrote:
       | I love these sorts of of things. I was fortunate enough to have
       | this fact mentioned in my stats book in undergrad, and later when
       | I was in Dublin and touring the Guinness brewery, they had a
       | small exhibit on Gosset (although Gossett was actually based out
       | of their brewery in England). Another fun fact that I learned in
       | organic chemistry was that Alexander Borodin (the Russian
       | composer who composed "In the Steppes of Central Asia" and
       | "Prince Igor") was only a composer in his spare time and was
       | actually professionally an organic chemist who was the co-
       | discoverer of the aldol reaction
       | 
       | https://en.wikipedia.org/wiki/Alexander_Borodin
        
       | gwern wrote:
       | I feel like OP unfortunately mostly misses the point of Gosset's
       | work and the t-test, in the usual way of people taught the
       | contemporary bastardization of Gosset/Fisher/Pearson/Neyman as
       | NHST. The important thing isn't that it lets you calculate some
       | _p_-value; the important thing is that it is a framework for
       | _decision-making_ , where you can trade off your false-positive
       | _and_ your false-negative rates to make the economically rational
       | decision:
       | https://www.tandfonline.com/doi/full/10.1080/00031305.2018.1...
       | 
       | In this case, because it is a well-understood problem with the
       | rates of bad batches easily established from the brewery's
       | records of testing & drinking, it lets the brewery decide on how
       | many 'off' batches it wants to risk in exchange for saving the
       | cost of a certain number of test-samples. You decide you want to
       | risk 1 bad batch in 100 for a false negative while rejecting 1
       | good batch in 20, then you need _n_ samples etc. And this
       | directly translates better measurements (by lowering variance,
       | eg. by blocking) into money: the lower the variance, the fewer
       | samples you need to achieve any given tradeoff, thereby saving
       | the brewery money on scrapped material or testing. The smaller
       | the better, hence Student's inability to use asymptotics or
       | approximations: they might be off by orders of magnitude. (He
       | would even try to do _n_ = 2 tests!)
       | 
       | Or they might be trying to tightly optimize alcohol content, to
       | avoid taxation for passing high-alcohol content thresholds, but
       | also avoid going too low to disappoint their customers, so
       | Student would explicitly calculate out scenarios, for example:
       | 
       | > Thus, Gosset concluded, "In order to get the accuracy we
       | require [that is, 10 to 1 odds with 0.5 accuracy], we must,
       | therefore, take the mean of [at least] four determinations." The
       | Guinness Board cheered. The Apprentice Brewer found an economical
       | way to assess the behavior of population parameters, using very
       | small samples.
       | 
       | (If you're thinking this sounds like a very subjective-Bayesian
       | decision-theory thing to write, you are right, although Student
       | would have rejected that, like most statisticians, and emphasized
       | that he was dealing with populations with known base rates, and
       | so nothing Bayesian was necessary; it was just a frequentist
       | decision-theory approach.)
        
         | nextos wrote:
         | That article you posted is excellent, thank you. Also relevant:
         | https://www.tandfonline.com/doi/abs/10.1080/01621459.1982.10...
        
       | killjoywashere wrote:
       | Another fun bit of biochemical history: Chaim Weizmann (1) was a
       | biochemist and staunch Zionist who gained the attention of First
       | Lord of the British Admiralty, Winston Churchill, for cultivating
       | a bacterium, Clostridium acetobutylicum, that could produce
       | acetone, which was in short supply and required for the
       | production of cordite, the key propellent in naval artillery
       | during World War I. In gratitude for Weizmann's contribution to
       | the war effort, George Lloyd asked him what Britain could do for
       | him, to which he replied "not for me, but for my people", which
       | begat the Balfour Declaration (2) establishing Britain's
       | commitment to provide a Home for the Jewish People (3).
       | 
       | (1) https://en.wikipedia.org/wiki/Chaim_Weizmann
       | 
       | (2) https://en.wikipedia.org/wiki/Balfour_Declaration
       | 
       | (3) It is essentially lost to history that the Balfour
       | Declaration also provided that the Palestinian people should not
       | be displaced. It is also mostly lost to history that there was a
       | great deal of politicking between Weizmann arriving in England
       | and his audience with George Lloyd, including a world tour
       | Weizmann orchestrated to promote one Albert Einstein.
        
         | rmason wrote:
         | I continue to be amazed the things you learn on HN. When I was
         | in Poland last summer I learned that a number of Jewish leaders
         | there between the two world wars advocated for a Jewish
         | homeland. But they were never able to convince the government
         | to publicly declare that support. Despite the fact at the time
         | Jews made up to 25% of Poland's population.
        
           | acidioxide wrote:
           | That's wrong. Polish state from 1926 onwards supported
           | Zionists (in the years 1926-1939, Poland was ruled by the
           | authoritarian Sanation movement).
        
             | rmason wrote:
             | Verified you are correct. Yet the museums I toured in both
             | Warsaw and Gdansk gave me the exact opposite impression.
        
         | riffraff wrote:
         | I've heard that tour was also accidentally the reason Einstein
         | became such a household name and face, tho I'm not so convinced
         | this is true.
        
         | jojobas wrote:
         | > Palestinian people should not be displaced
         | 
         | This was always wishful thinking at best, and more
         | realistically a lie.
        
           | cnlevy wrote:
           | Hmmm, those palestinian Arabs that made peace with the Jews
           | in the 1948 civil war are still there; see
           | 
           | https://en.m.wikipedia.org/wiki/Abu_Ghosh
           | 
           | https://en.m.wikipedia.org/wiki/Umm_al-Fahm
        
             | jojobas wrote:
             | Doesn't prove much. The Jews needed the fertile lands that
             | were very much settled, there were not all flocking there
             | for some desert.
             | 
             | Quoting Ben-Gurion:
             | 
             | A people which fights against the usurpation of its land
             | will not tire so easily. ... When we say that the Arabs are
             | the aggressors and we defend ourselves -- this is only half
             | the truth. ... [P]olitically we are the aggressors and they
             | defend themselves. The country is theirs, because they
             | inhabit it, whereas we want to come here and settle down,
             | and in their view we want to take away from them their
             | country.
        
         | cafard wrote:
         | Pedantry: David Lloyd George, Prime Minister during the later
         | part of WW I and for a time after.
        
         | hnbad wrote:
         | It also can't be overstated that Balfour himself was a staunch
         | racist and antisemite. His motivation for passing the
         | Declaration was at least in part the idea that if the UK gives
         | the Jews their own country, the UK will have a powerful ally in
         | the global Jewish conspiracy - and also encourage Jews to leave
         | the UK and not manipulate in its politics, culture and economy.
         | 
         | From Balfour's own writing in 1919 cited in his Wikipedia
         | article:
         | 
         | > [Zionism would] mitigate the age-long miseries created for
         | Western civilization by the presence in its midst of a Body
         | [the Jews] which it too long regarded as alien and even
         | hostile, but which it was equally unable to expel or to absorb.
         | 
         | He's the best example how support of Zionism and antisemitism
         | aren't mutually contradictory and can actually go hand in hand.
         | It's likewise often lost on people that there was a Zionist
         | project helping German Jews emigrate to Palestine with support
         | of the German government even after Hitler came to power,
         | although of course (like all migration) it ended with the
         | beginning of World War 2. This isn't to say the Nazis were fond
         | of this project but they didn't actively oppose it. They did
         | however pass laws requiring emigrating Jews to liquidate their
         | assets (i.e. sell off any businesses or property) and
         | significantly limiting the amount of wealth they could transfer
         | out of the country just like they later dispossessed (and
         | subsequently re-privatized) Jewish business owners and
         | confiscated their property during the Holocaust.
        
         | pjmorris wrote:
         | > "It is essentially lost to history"
         | 
         | In practice, this is clearly true. But it is not for lack of
         | trying, at least on some people's parts. I recently read 'A
         | Peace to End All Peace', David Fromkin, which goes in depth
         | into the background here, including Weizmann's role and the
         | making of the Balfour declaration. A passage from the book's
         | conclusion has stuck with me:
         | 
         | "It took Europe a millennium and a half to resolve its post-
         | Roman crisis of social and political identity: nearly a
         | thousand years to settle on the nation-state form of political
         | organization, and nearly five hundred years more to determine
         | which nations were entitled to be states. Whether civilization
         | would survive the raids and conflicts of rival warrior bands;
         | whether church or state, pope or emperor, would rule; whether
         | Catholic or Protestant would prevail in Christendom; whether
         | dynastic empire, national state, or city-state would command
         | fealty; and whether, for example, a townsman of Dijon belonged
         | to the Burgundian or to the French nation, were issues
         | painfully worked out through ages of searching and strife,
         | during which the losers--the Albigensians of southern France,
         | for example--were often annihilated. It was only at the end of
         | the nineteenth century, with the creation of Germany and Italy,
         | that an accepted map of western Europe finally emerged, some
         | 1,500 years after the old Roman map started to become obsolete.
         | The continuing crisis in the Middle East in our time may prove
         | to be nowhere near so profound or so long-lasting. But its
         | issue is the same: how diverse peoples are to regroup to create
         | new political identities for themselves after the collapse of
         | an ages-old imperial order to which they had grown accustomed."
        
           | insane_dreamer wrote:
           | One big difference with the Middle East is that the present
           | day borders were largely drawn up by foreign powers (England,
           | France) rather than evolving organically as they did in post-
           | Roman Europe. This is also a source of much of the ongoing
           | conflicts in the region including Israel/Palestine.
        
             | readthenotes1 wrote:
             | TBF, England and France took hundreds of years drawing up
             | their own borders between each other
        
         | leoc wrote:
         | I'm _not_ saying that this is a bad comment, or that it should
         | not have been made; but at the same time, I _am_ also a bit
         | wearied to see that the HN comment section has achieved a Time
         | To Palestine of 1 on this post.
        
           | jhardy54 wrote:
           | I'm _not_ saying that this is a bad comment, or that it
           | should not have been made; but at the same time, I am also a
           | bit wearied to see that the HN comment section has achieved a
           | Time To "Time To Palestine" Palestine of 2 on this post.
        
             | DiggyJohnson wrote:
             | These low effort posts are against the HN guidelines and
             | also not terribly clever.
        
         | racional wrote:
         | "Britain's commitment to provide a Home for the Jewish People
         | ... _in a place other than Europe_ " is the key detail that's
         | missing here.
         | 
         | It wasn't like they all got together and said, you know, it's
         | time we did the Jewish People a solid for once.
        
         | adhamsalama wrote:
         | Well, now I hate him. Thanks.
        
         | fsckboy wrote:
         | > _It is essentially lost to history that the Balfour
         | Declaration also provided that the Palestinian people should
         | not be displaced._
         | 
         | It is equally essentially lost on most people today, that the
         | Balfour Declaration did not create the state of Israel, nor did
         | the United Nations create it, nor the UK, nor the US, nor was
         | it post-war "resettlement plan" for displaced Jews by the
         | Allies. To the contrary, Jews at that time were barred from
         | entering the British Mandate.
         | 
         | Israelis declared themselves a state, similar to Americans
         | declaring themselves independent of Britain, against the wishes
         | of the UK. So it should also be noted that no "plan" for that
         | region called for a civil war: but a civil war broke out. So
         | it's revisionist shoehorning of the events that played out to
         | say that they didn't match such and such of various plans that
         | may have previously been made but never came to fruition. The
         | cis-Jordan Arabs were not treated worse by the Israelis than
         | they themselves were attempting to treat the Israelis; the
         | Arabs did lose the military conflict, however, after which they
         | displaced their own longstanding Jewish populations.
        
       | mcdonje wrote:
       | This story appears in "How to Measure Anything" by Douglas
       | Hubbard, which is worth a read if you're not already a stats and
       | decision theory whiz.
        
       | mcmoor wrote:
       | This Guiness connection is the core of the jokes when explaining
       | about t-test in Larry Gonick's History of Statistics.
        
       | colmmacc wrote:
       | My first year of working at AWS was in the "DUB1" site, which was
       | part of the Digital Hub (a tech and incubator space). As it
       | happened ... Amazon's small office was in William Sealy Gosset's
       | old laboratory, right beside St. Patrick's Tower where the
       | Guinness cooperage was. As a former statistics lecturer, I
       | excitedly told everyone I worked with how lucky we were, to
       | almost no reaction! Can you imagine?
       | 
       | A long time ago I submitted William Sealy Gosset as a suggestion
       | for commemoration with an Irish Postage Stamp; but nothing has
       | ever come of it. I hope some day he gets more recogonition.
        
       | iamcreasy wrote:
       | > Gosset recognized that this approach only worked with large
       | sample sizes, whereas small samples of hops wouldn't guarantee
       | that normal distribution. So he meticulously tabulated new
       | distributions for smaller sample sizes.
       | 
       | Does it mean Gosset stop before the distribution converging to
       | normal distribution?
        
       | VagabundoP wrote:
       | I went to school beside the Guinness brewery. The smell of the
       | hops brewing will remain with me forever.
       | 
       | However the school was dirt poor in many ways and they wouldn't
       | sponsor our school football team to buy some kit. This was back
       | in the 80's Ireland with massive unemployment and huge
       | emigration.
       | 
       | Ironically the school had a computer lab way beyond its time when
       | one ex-pupil donated a couple of Apple Macs and a dozen Apple
       | IIe's. That's were I cut my teeth on some - probably BASIC -
       | programming, learning myself.
       | 
       | On Topic - Guinness were always canny and ahead of their time.
       | Getting a job there was like winning the lotto, you were pretty
       | much made for life.
        
       | paperhatwriter wrote:
       | Comparing Guinness to an 'earthy milkshake' is one of the worst
       | things I've ever read.
        
       | hoseja wrote:
       | Artificially pumping beer full of nitrogen is kinda weird you
       | gotta admit.
        
         | mrob wrote:
         | The nitrogen is just a propellant for forcing the beer through
         | tiny holes that make the dissolved CO2 come out of solution in
         | tiny bubbles, which form a more stable foam. Very little
         | nitrogen actually dissolves into the beer.
        
       | aitchnyu wrote:
       | Louis Pasteur developed his technique for wine and beer, and milk
       | would benefit years later.
       | 
       | https://en.wikipedia.org/wiki/Pasteurization
        
       | westurner wrote:
       | The students' t distribution has a symmetric PDF (with no skew),
       | and thus you assume that the sample and/or population also have
       | such a PDF (Probability Distribution Function).
       | 
       | t statistic > History:
       | https://en.wikipedia.org/wiki/T-statistic#History
       | 
       | Students' t distribution:
       | https://en.wikipedia.org/wiki/Student%27s_t-distribution
       | 
       | "What are some alternatives to sample mean and t-test when
       | comparing highly skewed distributions"
       | https://www.quora.com/What-are-some-alternatives-to-sample-m... :
       | 
       | >> _the Kolmogorov-Smirnov two-sample test, which essentially
       | compares the empirical distribution functions of the two samples
       | without implicitly assuming
       | normality.http://en.wikipedia.org/wiki/Kolmogorov-Smirnov_test _
       | 
       | > _You may also be interested in the Wald-Wolfowitz runs test
       | (http://en.wikipedia.org/wiki/Wald-Wolfowitz_runs_test ) and the
       | Mann-Whitney test (http://en.wikipedia.org/wiki/Mann-Whitney_U
       | )._
       | 
       | Statistical Significance > Limitations, Challenges:
       | https://en.wikipedia.org/wiki/Statistical_significance#Limit...
       | 
       | Statistical hypothesis test > Criticism, Alternatives:
       | https://en.wikipedia.org/wiki/Statistical_hypothesis_test#Cr...
       | 
       | There are Multivariate Students' t distributions:
       | https://en.wikipedia.org/wiki/Multivariate_t-distribution
       | 
       | Matrix t distribution: https://en.wikipedia.org/wiki/Matrix_t-
       | distribution :
       | 
       | > _The generalized matrix t-distribution is the compound
       | distribution that results from an infinite mixture of a matrix
       | normal distribution with an inverse multivariate gamma
       | distribution placed over either of its covariance matrices._
       | 
       | But does a _matrix_ t-distribution describe nonlinear variance in
       | complex wave functions?
       | 
       | Quantum statistical mechanics:
       | https://en.wikipedia.org/wiki/Quantum_statistical_mechanics :
       | 
       | > _In quantum mechanics a statistical ensemble (probability
       | distribution over possible quantum states) is described by a
       | density operator S, which is a non-negative, self-adjoint, trace-
       | class operator of trace 1 on the Hilbert space H describing the
       | quantum system._
       | 
       | A Q12 question: How frequently are quantum density operators
       | described by a parametric t distribution?
        
       | the-mitr wrote:
       | I read this story and several other very interesting ones in this
       | great book detailing the history of evolution of modern
       | statistics -- The Lady Tasting Tea: How Statistics Revolutionized
       | Science in the Twentieth Century by David Salsburg.
       | 
       | The author himself personally met with several leading figures
       | that he describes. Highly recommended!
        
       | crispyambulance wrote:
       | Sometimes I wonder about the actual utility of the T-test
       | compared to just looking at a pair of boxplots, with jittered
       | points (or some other indication of the number of data points).
       | 
       | If it isn't plainly evident from the boxplots (assuming you've
       | got "enough points") do T-tests alone ever make a truly
       | compelling argument?
        
         | tmoravec wrote:
         | That would not be exactly scientific. T-test can be calculated
         | independently and verified.
        
           | _dain_ wrote:
           | But you still need to choose a significance threshold, which
           | is just an opinion. There's nothing "scientific" about
           | p=0.05; surely God loves p=0.06 almost as much.
        
       | rstuart4133 wrote:
       | The article left me wondering: how long after Guinness invented
       | the t-test did the hop growers invent p-hacking?
        
       ___________________________________________________________________
       (page generated 2024-05-27 23:02 UTC)