https://www.nature.com/articles/d41586-023-01780-w Skip to main content Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript. Advertisement Advertisement Nature * View all journals * Search * Log in * Explore content * About the journal * Publish with us * Subscribe * Sign up for alerts * RSS feed 1. nature 2. news 3. article * NEWS * 31 May 2023 AI intensifies fight against 'paper mills' that churn out fake research Text- and image-generating tools present a new hurdle for efforts to tackle the growing number of fake papers making their way into the academic literature. * Layal Liverpool 1. Layal Liverpool View author publications You can also search for this author in PubMed Google Scholar * Twitter * Facebook * Email You have full access to this article via your institution. Download PDF The hand of a businessman holding a pen and hovering above a doughnut chart printout on table. Artificial intelligence tools that can generate realistic images and text are making it difficult to detect fake research.Credit: Westend61/Alamy Advances in artificial intelligence (AI) are complicating publishers' efforts to tackle the growing problem of paper mills -- companies that produce fake scientific papers to order. Generative AI tools, including chatbots such as ChatGPT and image-generating software, provide new ways of producing paper-mill content, which could prove particularly difficult to detect. These were among the challenges discussed by research-integrity experts at a summit on 24 May, which focused on the paper-mill problem. [d41586-023] Paper-mill detector put to the test in push to stamp out fake science "The capacity of paper mills to generate increasingly plausible raw data is just going to be skyrocketing with AI," says Jennifer Byrne, a molecular biologist and publication-integrity researcher at New South Wales Health Pathology and the University of Sydney in Australia. "I have seen fake microscopy images that were just generated by AI," says Jana Christopher, an image-data-integrity analyst at the publisher Federation of European Biological Societies Press in Heidelberg, Germany. But being able to prove beyond suspicion that images are AI-generated remains a challenge, she says. Language-generating AI tools such as ChatGPT pose a similar problem. "As soon as you have something that can show that something's generated by ChatGPT, there'll be some other tool to scramble that," says Christopher. A stream of papers Anna Abalkina, a social scientist at the Free University of Berlin and an independent research-integrity analyst, suspects that there might be a delay in these AI tools becoming more apparent in the academic literature because of the length of the peer-review process. Perhaps in the next few months, "we will see the first stream of papers", she says. Byrne, Christopher and Abalkina were participants at the UNITED2ACT summit last week, which was convened by the Committee on Publication Ethics (COPE), a non-profit organization focused on ethics in academic publishing based in Eastleigh, UK, and the International Association of Scientific, Technical and Medical Publishers (STM), based in Oxford. The summit brought together international researchers, including independent research-integrity analysts, as well as representatives from funding bodies and publishers. [d41586-023] The fight against fake-paper factories that churn out sham science "It was the first time we had a group of people come together and co-create a set of actions which we're going to take forward to combat this problem," says Deborah Kahn, a trustee of COPE and a research-integrity consultant, based in London. The group intends to publish its joint action plan soon. When it comes to detecting paper-mill works, "there is absolutely an additional challenge which is posed by synthetic images, synthetic text, et cetera", says Joris van Rossum, programme director for STM Solutions, a subsidiary of STM. "There is a general realization that there is the potential of screening becoming more difficult," he says. AI assistance Kahn says that, although there will undoubtedly be positive uses of AI to support researchers writing papers, it will still be necessary to distinguish between legitimate papers written with AI and those that have been completely fabricated. "We have to really look at how we identify those things, and how we make sure that people have actually done the research. And there are various ways we can do that," she says. One strategy discussed during the summit was to require authors to provide the raw data from experiments, potentially with digital watermarks that would enable publishers to confirm that those data are genuine. [d41586-023] Hundreds of gibberish papers still lurk in the scientific literature Currently, requirements for submitting raw data vary significantly between publishers, says Christopher. Establishing a uniform set of requirements for the submission of raw data across publishers, taking into account differences between fields of research, could therefore be helpful, she says. Sabina Alam, director of publishing ethics and integrity at Taylor & Francis, a publisher based in Abingdon, UK, agrees but says that such standards will take time to implement. "I can't imagine it being an overnight flip, because the reality is many institutions don't actually have the resources to offer data-management infrastructure," she says. "We don't want to penalize actual research." Sharing information The summit also discussed other strategies for tackling the problem of paper mills more broadly, including organizing an awareness day or week for researchers, as well as identifying ways for publishers to share relevant information on suspected paper mills -- for example when publishers simultaneously receive submissions -- without breaching data-protection rules. STM is continuing to develop its own paper-mill detection software, while also collating resources on similar tools available elsewhere through its integrity hub. The apparent rise in paper mills increases demand for such techniques -- both for detecting fake papers at the point of submission and for identifying those that are already published. Taylor & Francis is among the publishers that are making use of such tools, and Alam says that a growing number of ethics cases -- instances of potential misconduct that are flagged for further investigation -- are being escalated to her team. Roughly half of these cases are because of paper mills, according to Alam. Her team saw the number of ethics cases increase more than tenfold from 2019 to 2022 -- and so far this year, there have been almost as many cases as during the whole of 2022. "It seems to have been commercialized and scaled up," she says. doi: https://doi.org/10.1038/d41586-023-01780-w Related Articles * [d41586-023] Paper-mill detector put to the test in push to stamp out fake science * [d41586-023] 'Papermill alarm' software flags potentially fake papers * [d41586-023] The fight against fake-paper factories that churn out sham science * [d41586-023] Hundreds of gibberish papers still lurk in the scientific literature * [d41586-023] What makes an undercover science sleuth tick? Fake-paper detective speaks out * [d41586-023] Publishers unite to tackle doctored images in research papers * [d41586-023] China's research-misconduct rules target 'paper mills' that churn out fake studies Subjects * Publishing * Scientific community Latest on: Publishing Turkey's researchers fear loss of freedom after Erdogan re-elected Turkey's researchers fear loss of freedom after Erdogan re-elected News 30 MAY 23 Anonymizing peer review makes the process more just Anonymizing peer review makes the process more just Career News 26 MAY 23 Social media: generative AI could harm mental health Correspondence 23 MAY 23 Scientific community Why ideas of 'planetary boundaries' must uphold environmental justice Why ideas of 'planetary boundaries' must uphold environmental justice Editorial 31 MAY 23 Why I'm leading Pacific Islands students in the fight on climate change Why I'm leading Pacific Islands students in the fight on climate change World View 30 MAY 23 Why the world needs more transparency on the origins of novel pathogens Why the world needs more transparency on the origins of novel pathogens Comment 30 MAY 23 Nature Careers Jobs * Recruitment of scientific research personnel in immunology, vaccinology and nanomedicine Professor, Associated Professor, Assistant Professor, Post-doctoral, Research Assistant Shenzhen, Guangdong, China Shenzhen Institutes of Advanced Technology (SIAT), Chinese Academy of Sciences(CAS) [] * High-level Talents globally and Director of Key Laboratory Recruitment of High-level Talents globally and Director of Key Laboratory for Green and High-Value Utilization of Salt Lake Resources Xining, Qinghai, China Qinghai Institute of Salt Lakes (QISL), Chinese Academy of Sciences [] * Recruitment of Dezhou (Qihe) Wheat Industry Research Institute of Shandong Agricultural University Recruitment of Outstanding Talents in the Field of Wheat Research at Home and Abroad by the Institute for Dezhou (Qihe) Wheat Industry Research of ... Shandong (CN); Shandong Agricultural University [] * Assistant/Associate Professor The Institute of Biomedical Engineering, National Tsing Hua University, seeks highly qualified individuals for 1 position in the assistant or assoc... Taiwan (TW) Institute of Biomedical Engineering [] * Distinguished Junior Fellowship The South Bay Interdisciplinary Science Center (SBISC) is one of the four core divisions of the Songshan Lake Materials Laboratory (SLA... Dongguan, Guangdong, China Songshan Lake Materials Laboratory (SLAB) [] You have full access to this article via your institution. Download PDF Related Articles * [d41586-023] Paper-mill detector put to the test in push to stamp out fake science * [d41586-023] 'Papermill alarm' software flags potentially fake papers * [d41586-023] The fight against fake-paper factories that churn out sham science * [d41586-023] Hundreds of gibberish papers still lurk in the scientific literature * [d41586-023] What makes an undercover science sleuth tick? Fake-paper detective speaks out * [d41586-023] Publishers unite to tackle doctored images in research papers * [d41586-023] China's research-misconduct rules target 'paper mills' that churn out fake studies Subjects * Publishing * Scientific community Advertisement Sign up to Nature Briefing An essential round-up of science news, opinion and analysis, delivered to your inbox every weekday. Email address [ ] [ ] Yes! Sign me up to receive the daily Nature Briefing email. I agree my information will be processed in accordance with the Nature and Springer Nature Limited Privacy Policy. Sign up * Close Nature Briefing Sign up for the Nature Briefing newsletter -- what matters in science, free to your inbox daily. Email address [ ] Sign up [ ] I agree my information will be processed in accordance with the Nature and Springer Nature Limited Privacy Policy. Close Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing Explore content * Research articles * News * Opinion * Research Analysis * Careers * Books & Culture * Podcasts * Videos * Current issue * Browse issues * Collections * Subjects * Follow us on Facebook * Follow us on Twitter * Subscribe * Sign up for alerts * RSS feed About the journal * Journal Staff * About the Editors * Journal Information * Our publishing models * Editorial Values Statement * Journal Metrics * Awards * Contact * Editorial policies * History of Nature * Send a news tip Publish with us * For Authors * For Referees * Language editing services * Submit manuscript Search Search articles by subject, keyword or author [ ] Show results from [All journals] Search Advanced search Quick links * Explore articles by subject * Find a job * Guide to authors * Editorial policies Nature (Nature) ISSN 1476-4687 (online) ISSN 0028-0836 (print) nature.com sitemap About Nature Portfolio * About us * Press releases * Press office * Contact us Discover content * Journals A-Z * Articles by subject * Nano * Protocol Exchange * Nature Index Publishing policies * Nature portfolio policies * Open access Author & Researcher services * Reprints & permissions * Research data * Language editing * Scientific editing * Nature Masterclasses * Nature Research Academies * Research Solutions Libraries & institutions * Librarian service & tools * Librarian portal * Open research * Recommend to library Advertising & partnerships * Advertising * Partnerships & Services * Media kits * Branded content Career development * Nature Careers * Nature Conferences * Nature events Regional websites * Nature Africa * Nature China * Nature India * Nature Italy * Nature Japan * Nature Korea * Nature Middle East * Privacy Policy * Use of cookies * Your privacy choices/Manage cookies * Legal notice * Accessibility statement * Terms & Conditions * Your US state privacy rights Springer Nature (c) 2023 Springer Nature Limited