[HN Gopher] Randomization in Controlled Experiments
       ___________________________________________________________________
        
       Randomization in Controlled Experiments
        
       Author : pykq
       Score  : 19 points
       Date   : 2026-03-17 15:01 UTC (3 days ago)
        
 (HTM) web link (queue.acm.org)
 (TXT) w3m dump (queue.acm.org)
        
       | hackingonempty wrote:
       | > So our farmer should compare a treatment group of turkeys that
       | watches TV with a control group that doesn't.
       | 
       | For human subjects if you do that you get an "A vs. A+B study"
       | which has a high probability of bias towards intervention B. Just
       | sitting and talking with someone in a white lab coat will improve
       | outcomes over those who do not.
       | 
       | You must compare to some sort of placebo and the closer it is to
       | the intervention you are testing the better. Testing a drug with
       | side effects against an inert placebo can break blinding and show
       | an effect for the drug that isn't there (Type I error.)
       | 
       | This experimental design is very common in the world of So-Called
       | Alternative Medicine (SCAM) e.g. "patients receiving conventional
       | treatment vs. patients receiving conventional treatment plus a
       | chiropractic technique." If you look for this you will find it
       | virtually every time a SCAM practitioner presents an RCT as
       | evidence for its efficacy.
        
       | madhadron wrote:
       | The ACM should probably have run this by a statistician before
       | publishing it. As another commenter pointed out it opens with bad
       | experimental design, then goes on about random allocation in a
       | way that makes it clear that the author doesn't understand the
       | subject well.
       | 
       | You don't need sequences of independent samples for assignment.
       | You need sequences that are independent of the sequence of
       | subjects being assigned. If you have a very small number of
       | possible sequences then you can't get independence by the pigeon
       | hole principle, but both 2^(64*N) (from N random 64 bit numbers)
       | or 2^64 - 1 (for N samples from a PRNG) are so large that it
       | doesn't matter.
       | 
       | The argument about a sample of seeds not producing a flat
       | histogram is also weird. Of course it doesn't. That's irrelevant
       | to whether uniformly sampling a sequence of N 64 bit numbers from
       | the space the PRNG provides produces independent sequences from
       | the subjects.
        
       ___________________________________________________________________
       (page generated 2026-03-20 23:01 UTC)