(C) PLOS One
This story was originally published by PLOS One and is unaltered.
 . . . . . . . . . .



Ending publication bias: A values-based approach to surface null and negative results [1]

['Stephen Curry', 'Department Of Life Sciences', 'Imperial College London', 'London', 'United Kingdom', 'Research On Research Institute', 'Eunice Mercado-Lara', 'Open Research Community Accelerator', 'San Francisco', 'California']

Date: 2025-09 

Abstract Sharing knowledge is a basic tenet of the scientific community, yet publication bias arising from the reluctance or inability to publish negative or null results remains a long-standing and deep-seated problem, albeit one that varies in severity between disciplines and study types. Recognizing that previous endeavors to address the issue have been fragmentary and largely unsuccessful, this Consensus View proposes concrete and concerted measures that major stakeholders can take to create and incentivize new pathways for publishing negative results. Funders, research institutions, publishers, learned societies, and the research community all have a role in making this an achievable norm that will buttress public trust in science.

Citation: Curry S, Mercado-Lara E, Arechavala-Gomeza V, Begley CG, Bernard C, Bernard R, et al. (2025) Ending publication bias: A values-based approach to surface null and negative results. PLoS Biol 23(9): e3003368. https://doi.org/10.1371/journal.pbio.3003368 Published: September 24, 2025 Copyright: © 2025 Curry et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. Funding: D.B. is supported by research funding from the National Institutes of Health (National Institute on Aging, National Institute on Neurological Disorders and Stroke). N.B. acknowledges funding through the National Institutes of Health (National Institute of General Medical Sciences - Grant Ref. R35GM141835. MGT receives partial funding from the Michael J Fox Foundation, the National Institutes of Health (National Institute on Neurological Disorders and Stroke), and the Indiana Collaborative Initiative for Talent Enrichment (INCITE) Lilly Endowment Fund. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. Competing interests: We have read the journal’s policy and the authors of this manuscript have the following competing interests: NB is an (unpaid) member of the Scientific and Medical Advisory Board of OpenRxiv. SB is a Senior Editor at Springer Nature. CC is an academic editor at PLOS Biology; ALJF is director of the not-for-profit Octopus Publishing CIC. RL is Managing Director of F1000. JGM is Editor in Chief of Neurology and receives salary support from the American Academy of Neurology for that role; in addition, he is an ex-officio member of the Board of Directors of the American Academy of Neurology. MM is President of the American Society for Cell Biology and a member of the editorial board for Molecular Biology of the Cell. PWS is Editor-in-Chief of microPublication Biology, published by the California Institute of Technology Library. MGT is a scientific consultant and/or advisor for Inmune Bio, iMetabolic Pharma, Novonordisk, Astex Pharmaceuticals, Arvinas, Longevity, Jaya, IMMvention, Nysnobio, Forward Therapeutics, Ventus, Merck, Weston Family Foundation, The Michael J. Fox Foundation for Parkinson’s Research, Bright Focus Foundation, the Parkinson’s Foundation, and New Vision Research for Charleston Conference on Alzheimer’s Disease (CCAD). She is co-inventor on a Xencor Inc. patent on dominant-negative TNF.

Introduction Scientific progress relies on the dissemination of knowledge so that others can build upon it. Yet not all scientific knowledge is disseminated [1]. Studies with ‘nonsignificant’ or ‘unexciting’ results often remain unpublished, a phenomenon known as ‘the file drawer problem’ or publication bias [2]. This bias persists despite decades of warnings against its consequences [3–8]. By its very nature, the extent of publication bias is difficult to ascertain, but the available data clearly indicate that null findings are underreported. For instance, according to recent studies, fewer than 2 in 100 articles on prognostic markers or animal models of stroke report null findings [9,10], a result that is unlikely to arise from a system that faithfully reports the outcome of experiments regardless of statistical significance. More concretely, published meta-analyses frequently show evidence of publication bias [11], and the introduction of the ‘registered report’ format, in which journals commit to publication before experimental outcomes are known, substantially increased the proportion of null fundings in some psychology journals [12]. Nevertheless, publication bias remains the subject of ongoing debate. In a wide-ranging critique of the so-called ‘crisis narrative’ that has emerged in recent years, Fanelli [13] recognizes the problem but cautions against universalizing it, citing the lack of evidence across many fields. Silvertown and Conway [14] argue that the impact of publication bias may have been overstated due to underestimates of the tendency for false positives to eventually succumb to further scrutiny. The extent of publication bias does seem to vary across disciplines. For example, surveys of meta-analyses suggest that publication bias is greater in some social science disciplines than it is in biomedical or physical sciences [11,15]. This variation can arise owing to any number of factors, ranging from choice of research methodology (e.g., observational studies, or research that is more descriptive, qualitative, or theoretical) to distinctive disciplinary norms or cultures. Importantly, both the magnitude and consequences of under-reporting of null findings also vary across disciplines. In biomedicine and clinical research, unreported null results can lead to patient-care risks, whereas in fields like economics or ecology, the societal impact of underreported null findings might be less obvious than the impact it has on research efficiency and advancing knowledge [16]. Notwithstanding the modulating effects of methodological and disciplinary variations, the consequences of publication bias can be severe. Unpublished null studies waste resources, slow the pace of science, and impede career advancement. Unwitting researchers are likely to expend time and money conducting similar experiments, not realizing that prior work has yielded null results: time and money that could have been spent pursuing new and more promising ideas. The failure to publish null findings also distorts the literature, resulting in exaggerated effect sizes [17–20], biased meta-analyses [11], inaccurate clinical trial results, flawed policy interventions [21–23], and the acceptance of false claims if incorrect ideas are not challenged [20,24]. Moreover, the rise of artificial intelligence (AI) models trained on incomplete data can magnify this problem, potentially contributing to misinformation [25] and undermining public trust in science. Publication bias is deeply rooted in scientific culture. Null studies are often perceived as less valuable, leading to harsher peer reviews [26–28] and lower citation rates [29,30]. Null studies are also commonly supposed to be unlikely to be accepted by higher-impact journals [31], further limiting their visibility and reinforcing biases against reporting such findings. Tenure and promotion systems frequently prioritize journal impact factors over methodological rigor [32,33], further discouraging researchers from sharing null results. Additional discouragement arises because reviewers of funding applications may focus on the perceived prestige of the publication outlet rather than the content of investigators’ prior work [34]. As a result, many investigators are reluctant to publish null studies despite the time and effort spent conducting experiments and other empirical studies [35–37]. Even apart from external drivers, investigators may, for good reason, lose confidence in a line of inquiry without having validated or controlled their findings to the rigorous degree that might be applied to a positive result, and therefore be less likely to write them up for publication. There are many ways for an experiment to go awry, and the constant pressure for researchers to focus on what they perceive to be their most productive lines of inquiry may lead them to set aside lines of research that have not yielded positive results. Journal practices can also exacerbate publication bias. A recent analysis (https://go.nih.gov/looOO3o) by the US National Institute of Neurological Disorders and Stroke (NINDS) found that 180 out of 215 neuroscience journals do not explicitly welcome null studies, while only 14 appeared to accept null studies without additional conditions (e.g., a higher burden of evidence than required for positive studies). Although newer scientific dissemination mechanisms such as study preregistration and Registered Reports [38–41] have shown promise in increasing the publication of null results [12,42], they are not universally applicable [43–47], despite recent efforts to extend their use to exploratory and observational studies [48,49]. Other more recent formats, such as micropublications [50] and modular publications [51], also offer promising avenues for sharing null studies, as do F1000 open research platforms and preprint and data repository platforms such as bioRxiv, arXiv, OSFPreprints, Zenodo, Figshare, and Dryad (bioRxiv even has a dedicated ‘Contradictory Results’ section), all of which can offer more frictionless avenues to dissemination than traditional journals [52,53]. However, these platforms still underrepresent null findings (https://go.nih.gov/looOO3o). Given the many and various factors at play in sustaining publication bias, what scope is there to prevent it from happening? In this Consensus View, we present a framework for practical action that seeks to enlist contributions from all the relevant stakeholders within the research ecosystem. Although our perspective is primarily biomedical, we believe there is a fresh opportunity for a wider discussion of the problem of publication bias across all disciplines and the construction of proportionate and effective measures to address it.

Methodology This ideas and reflections presented here were generated in discussions by participants of the Novel Approaches to Preventing Publication Bias Workshop (https://go.nih.gov/Y6iYPxU), which was run by the NINDS Office of Research Quality in May 2024 in Bethesda, Maryland, USA. This meeting aimed to assess progress and identify remaining barriers to disseminating null studies across the research ecosystem. Around 50 stakeholders were invited to ensure broad representation by sector (researchers and research institutions; traditional and nontraditional publishers; journal editors; funders; nonprofit organizations; industry; scientific societies), career stage (senior and early career researchers), and geography (United States of America, United Kingdom, Ireland, Germany, Spain, France, Mexico, Australia, plus virtual participation from Kenya). Although most attendees were drawn from biomedical fields, experts in psychology, economics, business, ecology, and public health were included to capture cross-disciplinary perspectives. The agenda combined plenary presentations with five sector-focused breakout sessions (researchers, research institutions, traditional and nontraditional publishers, scientific societies and nonprofit organizations, and funders and policymakers), each facilitated by a domain expert and an early career participant. Sessions were followed by a closing plenary in which all attendees synthesized key insights. Facilitators recorded breakout session notes using a shared template. These notes were consolidated in the closing plenary, then thematically coded by two independent analysts. Emergent themes were organized into the framework presented here, ensuring that domain-specific nuances and overarching patterns informed the final recommendations.

Values-based approach to system change To address the causes of publication bias, scholars from diverse disciplinary and geographical backgrounds gathered at a NINDS-hosted meeting in May 2024. They agreed that while scientists should be free to choose which questions to pursue, whatever the results of their research, the current scientific culture clearly incentivizes the production of ‘significant’ or positive results over methodological rigor [54–57]. To transform this culture, we argue that there is a need to shift away from valuing only positive or ‘exciting’ results towards prioritizing the importance of the research question and the quality of the research process, regardless of outcome [55,58,59]. We therefore propose a values-based approach [60–66] to reforming policies, activities, and incentives to reduce or eliminate the occurrence of publication bias (Fig 1). At its core, we believe this means removing barriers to sharing all knowledge (regardless of statistical significance or perceived impact) and enabling, incentivizing and centering the key academic values of transparency, accessibility, and openness. PPT PowerPoint slide

PNG larger image

TIFF original image Download: Fig 1. Values-based approach to reducing publication bias. Conceptual diagram illustrating the cyclic and iterative nature of the steps involved in seeking to end publication bias. https://doi.org/10.1371/journal.pbio.3003368.g001 Drawing principally on experience from biomedical and related science, technology, engineering and mathematics domains, but with relevance to social science, the humanities, and beyond, we therefore call on all sectors of the scientific enterprise to commit openly to the dissemination of all knowledge and to enact specific, practical changes to accomplish this goal within their respective disciplines. We propose a framework for action that includes improved incentive structures to reward transparency and rigor, the creation of simpler mechanisms for reporting null results, and collaboration among sectors of the scientific community to achieve this goal. Sustaining these changes will require reinforcement from a wide range of actors in the funding, delivery, and dissemination of research. Below, we outline concrete steps for different stakeholders to reduce publication bias. Openly commit to the value of disseminating all knowledge As a first step, we ask all scientific entities (including institutions, departments, core facilities, libraries, ethics committees, scientific societies, funders, journals, laboratory groups, and individual scientists) to reflect on the value of sharing and disseminating all knowledge and to hold internal conversations on why this is crucial for their missions. This is essential to winning internal buy-in from across organizations so that they are positioned to make credible and achievable commitments within their respective domains to sharing all knowledge and eliminating publication bias. Identify which practices align with sharing all knowledge Funders, research-performing organizations, and publishers should assess whether current practices facilitate or hinder transparent research processes. For example, do current practices of researcher assessment (e.g., graduation/degree requirements, promotion, and tenure policies) encourage dissemination of all high-quality research, including null results, or do they focus only on the most ‘exciting’ results (e.g., by heavily weighing bibliometrics or media coverage, which are dubious measures of research quality [67–70])? Does manuscript peer review give due weight to the questions being addressed and the methodological quality of the experiments when determining scientific merit and impact? Does grant review focus on the methodological rigor of investigators’ prior work rather than on the journals in which their papers were published? In many current research settings, the answer is no [55,71]. Explicitly outlining which practices should be modified enables a systematic approach to conceptualizing and implementing these changes, such as through new resources or incentives. Sharing research findings publicly should be a default part of the research workflow rather than an optional step that depends on study outcomes. Commitments to sharing all knowledge should be accompanied by a clear plan of action so that organizations can publicly be held accountable. Table 1 illustrates specific actions for different entities to consider. PPT PowerPoint slide

PNG larger image

TIFF original image Download: Table 1. High-priority interventions that promote the dissemination of all knowledge, including null studies, across research fields. https://doi.org/10.1371/journal.pbio.3003368.t001 While the ideas in Table 1 represent possible starting points for different stakeholders, we do not wish to be overly prescriptive in how to effect change. The most effective solutions are likely to be tailored to disciplinary norms and practices and will therefore require ground-level input, but we would nevertheless highlight some ideas for practical implementation by different stakeholders. Funders, for example, could pilot a ‘null-results summary’ program that requires a one-page report of null findings linked to project registries that are accessible to reviewers of subsequent funding applications; such reports could also usefully include mention of work packages or proposed research questions that were not pursued (e.g., because of unanticipated developments or shifts in priorities). Institutions could further support this effort by hosting discipline-specific ‘null-data clinics’, periodic forums where researchers present null results alongside methodological lessons learned. Publishers could place a more explicit emphasis on the review of methodological rigor when handling manuscripts and highlight high-quality papers reporting null findings to encourage submissions. They could also monitor acceptance rates of manuscripts containing positive or negative results to iteratively refine their guidelines for authors and reviewers. Enable and reinforce change Given their influence in the system, research funders (public and private) are uniquely positioned to lead the implementation of policies that incentivize research transparency. Funder policies that are adequately enforced provide strong but proportionate incentives to change researcher reporting behavior and institutional researcher assessment practices. Funders also have the resources to spearhead the development of new infrastructure for easy dissemination outside of traditional publishing venues, for example, by supporting platforms that enable simple and widespread sharing of concise reports of null findings or unfinished projects. Academic institutions will no doubt want to ensure their researchers are responding to funder incentives, but can synergize in other ways with efforts to address publication bias. For example, when assessing researchers, they should prioritize methodological quality in evaluating experimental outcomes. This could encourage a shift away from reliance on publication prestige and support outlets that emphasize methodological quality and openness. Enhancing education and awareness about effective knowledge sharing will further support these changes. More ambitious actions, such as creating new tools or platforms for publication and review of null results, will likely require careful staged implementation, piloting, and engagement with different stakeholders (e.g., funders, publishers, researchers, and reviewers). Only with proper incentives and well-designed processes will barriers to the publication of null results be overcome. Evaluate and iteratively improve interventions over time As with any effort to enact genuine reform, a crucial step will be to document, evaluate, and publicly share whether modifications to processes, policies, and practices increase research transparency. At the same time, reformers should be mindful to avoid any negative unintended consequences (e.g., inequity, inappropriate gaming of the system, or undue burden). There are a number of formal evaluation frameworks that may be helpful for guiding the change process [72–74]. These emphasize the importance of a systems-level perspective, collaborating with stakeholders to develop and contextualize the framework, and iterative incorporation of new insights to refine the intervention. Openness to critical evaluation throughout cycles of reform (Fig 1) will be essential if we are to make real progress in addressing publication bias.

Conclusions While sharing knowledge is a fundamental principle within the scientific community, publication bias has remained a tough nut to crack and requires renewed attention from the broader research community as part of wider debates about the health of the research and scholarly enterprise. Addressing the challenge of publication bias will not just enhance the progress of research, but also buttress the social contract that publicly funded research relies on for continued support. Our roadmap has a role for all stakeholders (funders, institutions, publishers, and researchers) in promoting knowledge sharing, research transparency, and rigor, but change will only happen if we are all willing to play our part.

Acknowledgments The authors would like to thank the organizers of the 2024 National Institute of Neurological Disorders and Stroke (NINDS) Novel Approaches to Preventing Publication Bias workshop, including Devon Crawford, Mariah Hoye, and Shai Silberberg, who contributed to early drafts of this manuscript. The authors would also like to thank all workshop participants for lively and informative discussions that shaped this paper, including Prachee Avasthi, Anna Hatch, Erin McKiernan, and Bodo Stern, who provided insightful comments on this manuscript. The content of this publication does not reflect the views or policies of the Department of Health and Human Services or of authors’ affiliated organizations, nor does mention of trade names, commercial products, or organizations imply endorsement by the United States Government.

[END]
---
[1] Url: https://journals.plos.org/plosbiology/article?id=10.1371/journal.pbio.3003368

Published and (C) by PLOS One
Content appears here under this condition or license: Creative Commons - Attribution BY 4.0.

via Magical.Fish Gopher News Feeds:
gopher://magical.fish/1/feeds/news/plosone/