[HN Gopher] Hack Your Way to Scientific Glory (2016)
       ___________________________________________________________________
        
       Hack Your Way to Scientific Glory (2016)
        
       Author : wjb3
       Score  : 45 points
       Date   : 2024-06-28 23:04 UTC (1 days ago)
        
 (HTM) web link (projects.fivethirtyeight.com)
 (TXT) w3m dump (projects.fivethirtyeight.com)
        
       | user_7832 wrote:
       | For anyone wondering about the title, it's about p-hacking -
       | picking and choosing variables to suit your objectives. Highly
       | recommend trying the mini-game, it's fun!
        
       | YossarianFrPrez wrote:
       | They say a photo is worth a thousand words, and here, this demo
       | is as convincing as a lot of the spilt ink regarding the
       | replication crisis and social psychology. As someone who pursuing
       | a PhD in the social sciences... This could be a useful tool for
       | seniors majoring in the social sciences / who are taking stats
       | classes.
        
       | paulpauper wrote:
       | Sigh. P hacking has long been a meme. Nicholas Cage and shark
       | attacks for example. P values are useful though if the results
       | seem at least plausible or can be explained by some underlying
       | process. Like a low p value between calorie restriction and
       | weight loss.
        
         | caddemon wrote:
         | Shark attacks isn't really p-hacking though, that's about
         | misinterpreting/misrepresenting a real correlation. P-hacking
         | is finding a fake correlation due to chance because you tested
         | a lot of different things in a smallish dataset, and then
         | reporting that correlation as meaningful without mentioning
         | your negative results or properly explaining your entire
         | process.
        
       | tbenst wrote:
       | I know the exercise was to p-hack, but instead I decided to one-
       | shot my attempt at the most reasonable model from first
       | principals:
       | 
       | - given that we are looking at a national scale, use only
       | national politicians
       | 
       | - use the components from Macroeconomics 101: exclude inflation
       | as that's on the Fed, exclude stocks as too conflated with FX and
       | international investing alternatives
       | 
       | - don't needlessly withhold data
       | 
       | Tried one hypothesis, so p-value of 0.04 is accurate. Still OK to
       | explore if you Bonferroni correct the p-Val afterwards
        
       | wjb3 wrote:
       | An excellent demonstration of how easy it is to manipulate data
       | to find significant results - it brings to light (again) the
       | pervasive issue of data dredging in research, where researchers,
       | intentionally or unintentionally, keep testing hypotheses until
       | they find something publishable, thereby undermining the
       | integrity of scientific findings and highlighting the importance
       | of preregistration and replication studies.
       | 
       | How can we shift the academic incentives away from "publish or
       | perish" toward promoting transparency and rigorous methodology in
       | research? Are any of the current attempts moving the needle?
        
         | jarsdel wrote:
         | Current attempts showing promise include initiatives like the
         | Center for Open Science and journals such as eLife and PLOS ONE
         | that emphasize methodological rigor over impact factor.
         | Additionally, policies from organizations like the National
         | Institutes of Health (NIH) mandating data sharing plans are
         | gradually moving the needle toward these goals.
        
       ___________________________________________________________________
       (page generated 2024-06-30 23:02 UTC)