https://www.nature.com/articles/d41586-025-01246-1 Skip to main content Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript. Advertisement Advertisement Nature * View all journals * Search * Log in * Explore content * About the journal * Publish with us * Subscribe * Sign up for alerts * RSS feed 1. nature 2. career column 3. article * CAREER COLUMN * 08 May 2025 P hacking -- Five ways it could happen to you Some data practices can lead to statistically dubious findings. Here's how to avoid them. By * Benjamin Tsang^0 1. Benjamin Tsang 1. Benjamin Tsang is a PhD candidate in the Department of Cell and Systems Biology at the University of Toronto, Canada. View author publications You can also search for this author in PubMed Google Scholar * Twitter * Facebook * Email You have full access to this article via your institution. A paper cut image of a multiple bell curve line graph Credit: MirageC / Getty It can happen so easily. You're excited about an experiment, so you sneak an early peek at the data to see if the P value -- a measure of statistical significance -- has dipped below the threshold of 0.05. Or maybe you've tried analysing your results in several different ways, hoping one will give you that significant finding. These temptations are common, especially in the cut-throat world of publish-or-perish academia. But giving in to them can lead to what scientists call P hacking. P hacking is the practice of tweaking the analysis or data to get a statistically significant result. In other words, you're fishing for a desirable outcome and reporting only the catches, while ignoring all the times you came up empty. It might get you a publication in the short term, but P hacking contributes to the reproducibility and replicability crisis in science by filling the literature with dubious or unfounded conclusions. Most researchers don't set out to cheat, but they could unknowingly make choices that push them towards a significant result. Here are five ways P hacking can slip into your research. Ending the experiment too early You might plan to gather 30 samples but find yourself running a quick analysis halfway through, just to see where things stand. If you notice a statistically significant difference after 15 samples, you might be inclined to stop the experiment early -- after all, you've found what you were looking for. [d41586-025] It's time to talk about ditching statistical significance But stopping an experiment once you find a significant effect but before you reach your predetermined sample size is classic P hacking. It's like declaring the winner of an election after polling just half the electorate: the result might not be representative of reality. What's the solution? Decide on the sample size or data-collection process ahead of time and stick to it, no matter how eager you are to see the results. Running experiments until you get a hit Another often-unintentional form of P hacking is repeating the experiment or analysis until you obtain a statistically significant result. Imagine you run an experiment and the outcome is insignificant. You try again with a new batch of samples -- still nothing. You repeat the study once more, and voila! P < 0.05. Success? Not quite. If you selectively report only the attempt that 'worked' and ignore those that didn't, you're engaging in P hacking by omission. As any gambler knows, if you roll the dice often enough, eventually you'll get the result you want by chance alone (not that I'm a gambler). The better approach is to report all the experimental replicates, including those that didn't work. Cherry-picking your results A less benign form of P hacking is selective reporting. Imagine you measure several outcomes or observe your effect at multiple time points -- for instance, testing a therapy's impact on recipients' blood pressure, cholesterol, weight and blood sugar regularly over an entire month. After analysing the data, you find that only one outcome -- say, blood sugar at week 3 -- showed a significant improvement. You might be tempted to highlight this one promising result and downplay the rest, or even omit them from your report. This is cherry-picking: by showing only the favourable data and ignoring everything else, you create a biased narrative. In this example, people might think the therapy worked because it lowered blood sugar at week 3, even though the overall data are not so rosy. Putting these data into the paper's supplementary material and continuing with the experiment on the basis of this one finding is also a no-no. You should report all relevant results, not just the ones that support the hypothesis. Science progresses faster when we know what doesn't work, as well as what does. Tweaking your data In data analysis, you often have to make judgements about what to include, what to exclude and how to report the data. P hacking can sneak in when those decisions are guided by the desire to achieve significance rather than by scientific reasoning. For example, you might notice an outlier in your data set. Including it in the analysis gives you a P value of 0.08, whereas excluding it brings P down to 0.03. Problem solved? Not quite. In these cases, it is best practice to go back to the original data or laboratory notes to determine whether experimental conditions could explain this outlier. Perhaps you pipetted double the amount of reagent into your sample, or construction work nearby during the time you were testing that animal affected its behaviour. Researchers can often rationalize their data-filtration decisions, and most of those decisions are warranted. But if the real motive is to turn an insignificant result into a significant one, it crosses into questionable territory. The key is to decide on data-filtering rules before looking at the results. If, for some reason, you have to make a change after data collection, explain that -- and say why. Enjoying our latest content? Login or create an account to continue * Access the most recent journalism from Nature's award-winning team * Explore the latest features & opinion covering groundbreaking research Access through your institution or Sign in or create an account # Continue with Google # Continue with ORCiD doi: https://doi.org/10.1038/d41586-025-01246-1 This is an article from the Nature Careers Community, a place for Nature readers to share their professional experiences and advice. Guest posts are encouraged. Competing Interests The author declares no competing interests. Related Articles * [d41586-025] 'Spell-checker for statistics' reduces errors in the psychology literature * [d41586-025] One statistical analysis must not rule them all * [d41586-025] Smart software spots statistical errors in psychology papers * Retiring statistical significance would give bias a free pass * [d41586-025] It's time to talk about ditching statistical significance * [d41586-025] Collection: Scientific data Subjects * Careers * Research data * Authorship Latest on: Careers What an angry exchange with a reviewer taught me about arrogance and humility What an angry exchange with a reviewer taught me about arrogance and humility Career Column 13 MAY 25 Six striking images showcase scientific fieldwork Six striking images showcase scientific fieldwork Career Feature 13 MAY 25 My PhD adviser was fired and I was collateral damage. I learnt how to build resilience into graduate school My PhD adviser was fired and I was collateral damage. I learnt how to build resilience into graduate school World View 13 MAY 25 Research data Renewal of NIH grants linked to more innovative results, study finds Renewal of NIH grants linked to more innovative results, study finds News 08 MAY 25 How to protect research data How to protect research data Nature Index 30 APR 25 Adversarial testing of global neuronal workspace and integrated information theories of consciousness Adversarial testing of global neuronal workspace and integrated information theories of consciousness Article 30 APR 25 Authorship The do's and don'ts of scientific image editing The do's and don'ts of scientific image editing Technology Feature 29 APR 25 Can peer-reviewed podcasts fast-track science? Can peer-reviewed podcasts fast-track science? Nature Index 10 APR 25 Gender gap in research publishing is improving -- slowly Gender gap in research publishing is improving -- slowly Nature Index 26 MAR 25 Nature Careers Jobs * Tenured/Tenure-Track Faculty Recruitment for Genome Editing Research Program The Dept of Ophthalmology at UC Irvine SOM is seeking accomplished Scientists or Physician-Scientists and invites applications. Irvine, California (US) UC Irvine Department of Ophthalmology [] * Associate Professor/Professor Research Professor in Molecular Epidemiology to initiate research focused on cohort studies to identify molecular risk factors for chronic disease Diepenbeek (Commune), Hasselt (BE) Biomedical Research Institute - Hasselt University [] * Two postdoctoral positions - Studying Climate Change with plant ecophysiology and NMR spectroscopy The Department of Medical Biochemistry and Biophysics at Umea University, Sweden, invites applicants for 2 (two) two-year Postdoctoral positions Umea, Sweden Umea University [] * NIMS Postdoctral researcher We are seeking a postdoctoral researcher to join our team in the development of advanced polymer materials designed to promote material circulation. Tsukuba, Ibaraki (JP) National Institute for Materials Science (NIMS) [] * Deputy Directors of the State Key Lab at Institute of Automation, Chinese Academy of Sciences Chinese Academy of Sciences is seeking candidates for Deputy Directors of the State Key Laboratory of Multimodal Artificial Intelligence Systems. Beijing, China Institute of Automation, Chinese Academy of Sciences [] Related Articles * [d41586-025] 'Spell-checker for statistics' reduces errors in the psychology literature * [d41586-025] One statistical analysis must not rule them all * [d41586-025] Smart software spots statistical errors in psychology papers * Retiring statistical significance would give bias a free pass * [d41586-025] It's time to talk about ditching statistical significance * [d41586-025] Collection: Scientific data Subjects * Careers * Research data * Authorship Advertisement Sign up to Nature Briefing An essential round-up of science news, opinion and analysis, delivered to your inbox every weekday. Email address [ ] [ ] Yes! Sign me up to receive the daily Nature Briefing email. I agree my information will be processed in accordance with the Nature and Springer Nature Limited Privacy Policy. Sign up Close Nature Briefing Sign up for the Nature Briefing newsletter -- what matters in science, free to your inbox daily. Email address [ ] Sign up [ ] I agree my information will be processed in accordance with the Nature and Springer Nature Limited Privacy Policy. Close Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing Explore content * Research articles * News * Opinion * Research Analysis * Careers * Books & Culture * Podcasts * Videos * Current issue * Browse issues * Collections * Subjects * Follow us on Facebook * Follow us on Twitter * Subscribe * Sign up for alerts * RSS feed About the journal * Journal Staff * About the Editors * Journal Information * Our publishing models * Editorial Values Statement * Journal Metrics * Awards * Contact * Editorial policies * History of Nature * Send a news tip Publish with us * For Authors * For Referees * Language editing services * Submit manuscript Search Search articles by subject, keyword or author [ ] Show results from [All journals] Search Advanced search Quick links * Explore articles by subject * Find a job * Guide to authors * Editorial policies Nature (Nature) ISSN 1476-4687 (online) ISSN 0028-0836 (print) nature.com sitemap About Nature Portfolio * About us * Press releases * Press office * Contact us Discover content * Journals A-Z * Articles by subject * protocols.io * Nature Index Publishing policies * Nature portfolio policies * Open access Author & Researcher services * Reprints & permissions * Research data * Language editing * Scientific editing * Nature Masterclasses * Research Solutions Libraries & institutions * Librarian service & tools * Librarian portal * Open research * Recommend to library Advertising & partnerships * Advertising * Partnerships & Services * Media kits * Branded content Professional development * Nature Careers * Nature Conferences Regional websites * Nature Africa * Nature China * Nature India * Nature Italy * Nature Japan * Nature Middle East * Privacy Policy * Use of cookies * Your privacy choices/Manage cookies * Legal notice * Accessibility statement * Terms & Conditions * Your US state privacy rights Springer Nature (c) 2025 Springer Nature Limited