https://www.nature.com/articles/d41586-025-00648-5 Skip to main content Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript. Advertisement Advertisement Nature * View all journals * Search * Log in * Explore content * About the journal * Publish with us * Subscribe * Sign up for alerts * RSS feed 1. nature 2. news 3. article * NEWS * 07 March 2025 AI tools are spotting errors in research papers: inside a growing movement Study that hyped the toxicity of black plastic utensils inspires projects that use large language models to check papers. By * Elizabeth Gibney 1. Elizabeth Gibney View author publications You can also search for this author in PubMed Google Scholar * Twitter * Facebook * Email A large stack of papers and folders with coloured tabs. Two new AI tools check for errors in research papers including in the calculations, methodology and references.Credit: Jose A. Bernat Bacete/Getty Late last year, media outlets worldwide warned that black plastic cooking utensils contained worrying levels of cancer-linked flame retardants. The risk was found to be overhyped - a mathematical error in the underlying research suggested a key chemical exceeded the safe limit when in fact it was ten times lower than the limit. Keen-eyed researchers quickly showed that an artificial intelligence (AI) model could have spotted the error in seconds. The incident has spurred two projects that use AI to find mistakes in the scientific literature. The Black Spatula Project is an open-source AI tool that has so far analysed around 500 papers for errors. The group, which has around eight active developers and hundreds of volunteer advisers, hasn't made the errors public yet; instead, it is approaching the affected authors directly, says Joaquin Gulloso, an independent AI researcher based in Cartagena, Colombia, who helps to coordinate the project. "Already, it's catching many errors," says Gulloso. "It's a huge list. It's just crazy." The other effort is called YesNoError and was inspired by the Black Spatula Project, says founder and AI entrepreneur Matt Schlicht. The initiative, funded by its own dedicated cryptocurrency, has set its sights even higher. "I thought, why don't we go through, like, all of the papers?" says Schlicht. He says that their AI tool has analysed more than 37,000 papers in two months. Its website flags papers in which it has found flaws - many of which have yet to be verified by a human, although Schlicht says that YesNoError has a plan to eventually do so at scale. Both projects want researchers to use their tools before submitting work to a journal, and journals to use them before they publish, the idea being to avoid mistakes, as well as fraud, making their way into the scientific literature. The projects have tentative support from academic sleuths who work in research integrity. But there are also concerns over the potential risks. How well the tools can spot mistakes, and whether their claims have been verified, must be made clear, says Michele Nuijten, a researcher in metascience at Tilburg University in the Netherlands. "If you start pointing fingers at people and then it turns out that there was no mistake, there might be reputational damage," she says. Others add that although there are risks and the projects need to be cautious about what they claim, the goal is the right one. It is much easier to churn out shoddy papers than it is to retract them, says James Heathers, a forensic metascientist at Linnaeus University in Vaxjo, Sweden. As a first step, AI could be used to triage papers for further scrutiny, says Heathers, who has acted as a consultant for the Black Spatula Project. "It's early days, but I'm supportive" of the initiatives, he adds. AI sleuths Many researchers have dedicated their careers to spotting integrity concerns in papers - and tools to check certain facets of papers already exist. But advocates hope that AI could carry out a wider range of checks in a single shot and handle a larger volume of papers. Both the Black Spatula Project and YesNoError use large language models (LLMs) to spot a range of errors in papers, including ones of fact as well as in calculations, methodology and referencing. The systems first extract information, including tables and images, from the papers. They then craft a set of complex instructions, known as a prompt, which tells a 'reasoning' model -- a specialist type of LLM -- what it is looking at and what kinds of error to hunt for. The model might analyse a paper multiple times, either scanning for different types of error each time, or to cross-check results. The cost of analysing each paper ranges from 15 cents to a few dollars, depending on the length of the paper and the series of prompts used. The rate of false positives, instances when the AI claims an error where there is none, is a major hurdle. Currently, the Black Spatula Project's system is wrong about an error around 10% of the time, says Gulloso. Each alleged error must be checked with experts in the subject, and finding them is the project's greatest bottleneck, says Steve Newman, the software engineer and entrepreneur who founded the Black Spatula Project. So far, Schlicht's YesNoError team has quantified the false positives in only around 100 mathematical errors that the AI found in an initial batch of 10,000 papers. Of the 90% of authors who responded to Schlicht, all but one agreed that the error detected was valid, he says. Eventually, YesNoError is planning to work with ResearchHub, a platform which pays PhD scientists in cryptocurrency to carry out peer review. When the AI has checked a paper, YesNoError will trigger a request to verify the results, although this has not yet started. False positives Enjoying our latest content? Login or create an account to continue * Access the most recent journalism from Nature's award-winning team * Explore the latest features & opinion covering groundbreaking research Access through your institution or Sign in or create an account # Continue with Google # Continue with ORCiD doi: https://doi.org/10.1038/d41586-025-00648-5 Reprints and permissions Subjects * Technology * Software * Machine learning Latest on: Technology Ice-hunting Moon lander runs into trouble -- leaving scientists in suspense Ice-hunting Moon lander runs into trouble -- leaving scientists in suspense News 07 MAR 25 Boosting science: 'Give foreign STEM students ten-year visas' Boosting science: 'Give foreign STEM students ten-year visas' Career Q&A 03 MAR 25 ChatGPT for students: learners find creative new uses for chatbots ChatGPT for students: learners find creative new uses for chatbots Technology Feature 03 MAR 25 Software What are the best AI tools for research? Nature's guide What are the best AI tools for research? Nature's guide News 17 FEB 25 Mapping cells through time and space with moscot Mapping cells through time and space with moscot Article 22 JAN 25 Accurate predictions on small data with a tabular foundation model Accurate predictions on small data with a tabular foundation model Article 08 JAN 25 Machine learning Artificial intelligence could boost eye care in low-income countries Artificial intelligence could boost eye care in low-income countries Outlook 05 MAR 25 How much energy will AI really consume? The good, the bad and the unknown How much energy will AI really consume? The good, the bad and the unknown News Feature 05 MAR 25 AI algorithm helps telescopes to pivot fast towards gravitational-wave sources AI algorithm helps telescopes to pivot fast towards gravitational-wave sources News & Views 05 MAR 25 Nature Careers Jobs * Faculty Positions of Artificial Intelligence and Data Science in Medicine at CIMR, Beijing At all ranks who focus on research in Artificial Intelligence, Data Science, and Machine Learning in Health Care and Medical Sciences. Beijing, China The Chinese Institutes for Medical Research (CIMR), Beijing [] * Become an Assistant or Associate Research Professor (PSO) in Cellular, Molecular and Synaptic Neuros Universite de Montreal's Faculty of Medicine is one of the world's leading French-language medical faculties. In both teaching and research, its mi... Montreal, Quebec (CA) Universite de Montreal [] * Principal Scientist Neuroscience Group IPI is seeking a Principal Scientist to drive research at the intersection of molecular neuroscience and protein science. Boston, Massachusetts (US) Institute for Protein Innovation [] * Cancer Immunology Postdoctoral Fellow Tampa, Florida H. Lee Moffitt Cancer Center & Research Institute [] * Postdoc in Cancer Data Science - Sottoriva Lab, Computational Biology Research Centre APPLICATION CLOSING DATE: April 13th, 2025 The integration of Artificial Intelligence, mathematical modelling, and advanced computational simulati... Milan (IT) Human Technopole [] Subjects * Technology * Software * Machine learning Advertisement Sign up to Nature Briefing An essential round-up of science news, opinion and analysis, delivered to your inbox every weekday. Email address [ ] [ ] Yes! Sign me up to receive the daily Nature Briefing email. I agree my information will be processed in accordance with the Nature and Springer Nature Limited Privacy Policy. Sign up Close Nature Briefing Sign up for the Nature Briefing newsletter -- what matters in science, free to your inbox daily. Email address [ ] Sign up [ ] I agree my information will be processed in accordance with the Nature and Springer Nature Limited Privacy Policy. Close Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing Explore content * Research articles * News * Opinion * Research Analysis * Careers * Books & Culture * Podcasts * Videos * Current issue * Browse issues * Collections * Subjects * Follow us on Facebook * Follow us on Twitter * Subscribe * Sign up for alerts * RSS feed About the journal * Journal Staff * About the Editors * Journal Information * Our publishing models * Editorial Values Statement * Journal Metrics * Awards * Contact * Editorial policies * History of Nature * Send a news tip Publish with us * For Authors * For Referees * Language editing services * Submit manuscript Search Search articles by subject, keyword or author [ ] Show results from [All journals] Search Advanced search Quick links * Explore articles by subject * Find a job * Guide to authors * Editorial policies Nature (Nature) ISSN 1476-4687 (online) ISSN 0028-0836 (print) nature.com sitemap About Nature Portfolio * About us * Press releases * Press office * Contact us Discover content * Journals A-Z * Articles by subject * protocols.io * Nature Index Publishing policies * Nature portfolio policies * Open access Author & Researcher services * Reprints & permissions * Research data * Language editing * Scientific editing * Nature Masterclasses * Research Solutions Libraries & institutions * Librarian service & tools * Librarian portal * Open research * Recommend to library Advertising & partnerships * Advertising * Partnerships & Services * Media kits * Branded content Professional development * Nature Careers * Nature Conferences Regional websites * Nature Africa * Nature China * Nature India * Nature Italy * Nature Japan * Nature Middle East * Privacy Policy * Use of cookies * Your privacy choices/Manage cookies * Legal notice * Accessibility statement * Terms & Conditions * Your US state privacy rights Springer Nature (c) 2025 Springer Nature Limited