(C) PLOS One
This story was originally published by PLOS One and is unaltered.
 . . . . . . . . . .



Widespread misidentification of scanning electron microscope instruments in the peer-reviewed materials science and engineering literature [1]

['Reese A. K. Richardson', 'Department Of Engineering Sciences', 'Applied Mathematics', 'Northwestern University', 'Evanston', 'Illinois', 'United States Of America', 'Department Of Molecular Biosciences', 'Jeonghyun Moon', 'Spencer S. Hong']

Date: 2025-09 

Abstract Materials science and engineering (MSE) research has, for the most part, escaped the doubts raised about the reliability of the scientific literature by recent large-scale replication studies in psychology and cancer biology. However, users on post-publication peer review sites have recently identified dozens of articles where the make and model of the scanning electron microscope (SEM) listed in the text of the paper does not match the instrument’s metadata visible in the images in the published article. In order to systematically investigate this potential risk to the MSE literature, we develop a semi-automated approach to scan published figures for this metadata and check it against the SEM instrument identified in the text. Starting from an exhaustive set of 1,067,108 articles published since 2010 in 50 journals with impact factors ranging from 2 to 24, we identify 11,314 articles for which SEM manufacturer and model can be identified in an image’s metadata. For 21.2% of those articles, the image metadata does not match the SEM manufacturer or model listed in the text and, for another 24.7%, at least some of the instruments used in the study are not reported. We find that articles with SEM misidentification are more likely to have existing observations of improprieties made on post-publication peer review site PubPeer than other MSE articles and that a subset of these articles within the subfield of electrochemistry are more likely to incorrectly estimate the optical band gap if the article features SEM misidentification. This suggests that SEM misidentification may be a tractable signature for flagging problematic MSE articles. Unexplained patterns common to many of these articles suggest the involvement of paper mills, organizations that mass-produce, sell authorship on, and publish fraudulent scientific manuscripts at scale.

Citation: Richardson RAK, Moon J, Hong SS, Amaral LAN (2025) Widespread misidentification of scanning electron microscope instruments in the peer-reviewed materials science and engineering literature. PLoS One 20(7): e0326754. https://doi.org/10.1371/journal.pone.0326754 Editor: Zeheng Wang, Commonwealth Scientific and Industrial Research Organisation, AUSTRALIA Received: March 11, 2025; Accepted: June 4, 2025; Published: July 17, 2025 Copyright: © 2025 Richardson et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited. Data Availability: All data presented in this work which can be published according to the copyright information and text and data mining licenses of each publisher surveyed and which is not under license from the PubPeer Foundation is available in Supplementary Materials. Data from OpenAlex is publicly available. Identifiers and labels of each article analyzed are available at https://doi.org/10.17605/OSF.IO/6C5RF. Funding: R.A.K.R. was supported in part by the National Institutes of Health Training Grant [T32GM008449] through Northwestern University’s Biotechnology Training Program. R.A.K.R. gratefully acknowledges funding from the Dr. John N. Nicholson fellowship from Northwestern University; Moderna Inc., Identifying bias and improving reproducibility in RNA-seq computational pipelines. J.M. gratefully acknowledges funding from the Weinberg College Baker Program in Undergraduate Research. S.S.H. gratefully acknowledges support from the Ryan Fellowship and the International Institute for Nanotechnology at Northwestern University. L.A.N.A. gratefully acknowledge funding from SCISIPBIO: a data-science approach to evaluating the likelihood of fraud and error in published studies [1956338]. Competing interests: The authors declare that they have no competing interests.

Introduction Trust in published peer-reviewed results is critical to the efficient operation of the scientific enterprise. It is implicitly assumed in the peer-review process that the authors of a study have, to the best of their abilities, carefully and systematically reviewed their work for potential flaws and that no deception is being attempted. Such assumptions are sometimes spectacularly violated, as in the infamous case of J. H. Schön, but are nonetheless thought to hold broadly. It is thus not surprising to see the widespread concern raised by recent large-scale replication studies [1–3] that have found many published results in the social sciences and biology to be irreproducible. Despite Schön’s case and other recent high-profile retractions in superconductivity research [4], lack of replicability of results in the physico-chemical sciences and engineering has not been broadly regarded as concerning. However, two studies have recently reported widespread irreproducibility in the characterization of metal-organic frameworks [5] and hydrogen storage materials [6]. Amid a broader “reproducibility crisis" [7] and an exponentially-growing scientific literature [8], there has been an increasing interest in developing approaches for detection of markers of irreproducibility, errors and impropriety at-scale. For instance, Labbé and colleagues developed Seek & Blastn to rapidly screen publications for misidentified nucleotide reagents [9–11]. Bik and colleagues manually screened over 20,000 published articles in the life sciences for image duplication [12], finding that 3.8% of published articles contained inappropriately duplicated figure elements. Semi-automated approaches that accelerate the discovery of image integrity issues have also been developed [13–15]. Several publishers now employ automated methods for detecting image duplication in unpublished manuscripts [16]. In an effort to extend the development of automated or semi-automated tools in the context of the physico-chemical sciences and engineering, we study here the reporting of scanning electron microscope (SEM) instrumentation in scientific papers in material science and engineering, broadly defined. SEMs are a critical tool for the characterization of samples [17, 18]. Tens of thousands of articles using SEM are published annually (Fig 1a and S1 Fig). Studies reporting on original research articles will identify the manufacturer, model and operating parameters of the SEM used — e.g. “samples were observed with a Philips XL30 field emission scanning electron microscope at an accelerating voltage of 10 kV." Many times, the published images obtained with the SEM will include the instrument’s auto-generated “banner" that discloses experimental metadata (Fig 1b). Depending on the SEM manufacturer, this banner can include information on the instrument’s manufacturer and model (Fig 1c), various operating parameters of the instrument, and the facility that operates the instrument. PPT PowerPoint slide

PNG larger image

TIFF original image Download: Fig 1. We developed a semi-automated pipeline for detection of misidentified SEM instruments in the MSE literature and tested it on published articles in 50 journals. a, Annual count of articles featuring SEM based on Scopus search results. b, SEM image featuring an auto-generated metadata banner from a TESCAN MAIA3 SEM, after [19]. c, Illustrative metadata banner snippets allowing for identification of the manufacturer and/or model of instrument used. d, Publication timeline of the 1,067,108 articles extracted from 50 journals between 2010 and early 2023. SEM image were present in 174,046 articles (16.3%). https://doi.org/10.1371/journal.pone.0326754.g001 Recently, several users on the post-publication peer review site PubPeer [20] have documented dozens of instances of articles where the SEM instrument listed within the manuscript’s text does not match the instrument’s metadata extractable from the published images [21]. Such extreme carelessness suggests that other things may be amiss with the study. It is difficult to imagine such a mistake being made in good-faith preparation of an article. However, if articles are hastily mass-produced, or the article text was plagiarized from one source and the article images from another, these would be among the likely details to be missed. Indeed, PubPeer commenters often note other inconsistencies in these articles that directly call into question the reliability of the findings or the provenance of the article. To systematically investigate this matter, we developed a semi-automated pipeline for detection of misreporting of SEM instrumentation in published research articles. We deployed our method on more than a million articles from across 50 journals published by 4 different publishers, nearly 175 thousand of which included SEM images (Fig 1d). While we were only able to extract SEM metadata for approximately 11 thousand articles, we found that for 21.2% of these articles the image metadata did not match the SEM manufacturer or model listed in the text of the manuscript. For another 24.7%, at least some of the SEM instruments used in the study were not disclosed.

Data and methods In order to produce a precise estimate of the rate at which misidentification of SEM instrumentation occurs in materials science and engineering literature overall, one would need to obtain a representative sample of journals appropriately stratified by publisher, impact, reputation, subfield, author characteristics, and so on. Unfortunately, several factors precluded achieving this goal. First, several large publishers of high-profile materials science journals have established ‘data mining’ licensing terms and conditions that limit at-scale downloading to only open-access content and to PDF files. Second, many publishers severely limit request rates, precluding comprehensive downloading of the articles they publish. Third, the sensitivity of automated approaches is dependent on the image size of published scientific images. Unless images are sufficiently large and high-resolution, metadata banners cannot be automatically identified. The size of images published as figures has changed substantially over time and varies considerably across publishers (Fig 2 and S2 Fig). Finally, SEM instrument misidentification is only detectable where metadata banners are visible in the published figures. In the vast majority of materials science articles where metadata banners are not visible, rates of instrument misidentification are entirely unknowable. PPT PowerPoint slide

PNG larger image

TIFF original image Download: Fig 2. Comparison of the sizes of images, in terms of image file height in pixels, from which we were able to extract brandmarks using optical character recognition (OCR) (red boxes) versus images for which we were not (white boxes). a, For all years, image files for which we were able to extract brandmarks were larger. b, Comparison across 4 publishers of the sizes of images for which we were able to extract brandmarks (red boxes) versus images for which we were not (white boxes). Except studies published by the Public Library of Science, images for which we were able to extract brandmarks were significantly larger. Center line shows median, boxes show inter-quartile range, whiskers show 2.5th percentile and 97.5th percentile. n.s. = p > 0.05, * = p < 0.05, ** = p < 0.01 and *** = p < 0.001 by two-sided Mann-Whitney U test. Additional analysis shown in S2 Fig. https://doi.org/10.1371/journal.pone.0326754.g002 Due to these constraints, we focused on fifty journals from four publishers that (i) are indexed in OpenAlex, a free and open-source index of the scientific literature [22], (ii) span a broad range of impact factors, (iii) span a broad range of subfields within materials science and engineering and (iv) for which there is evidence of research impropriety based on existing comments on PubPeer. All content was downloaded following the relevant data mining terms, conditions and licenses specified by the publishers. We focus heavily on Elsevier journals due to the publisher’s relatively permissive data mining terms and conditions. We also included journals published by three additional publishers: Frontiers Media, Springer Nature, and the Public Library of Science. Several journals were multidisciplinary in scope, while others were focused on chemistry, materials science, chemical engineering, optics, biomedicine, and construction engineering. All but two journals were indexed by Web of Science [23] with 2022 Journal Impact Factors (IFs) ranging between 2.1 and 24.2. Journal characteristics are listed in S1 Table. We crawled the webpage of all 1,067,108 articles published in the 50 journals since 2010 using the digital object identifiers (DOIs) obtained from OpenAlex (downloaded Feburary 27, 2023). For each article, we scanned the figure captions for terms indicative of SEM usage such as “sem micrograph" and “scanning electron micro" (see S2 Table for a complete list). This filtering step identified 174,046 articles with an average of 1.5 SEM-related figures per article. We downloaded the largest available versions of each of these figures. We next used Tesseract 4 [24] optical character recognition (OCR) at 1.0, 1.5 and 2.0 scale to process each of the obtained 267,603 figures. We then compared the extracted strings against a manually curated set of brandmarks and model numbers known to appear in SEM metadata banners (Fig 1c). Examples include “TESCAN”, “KYKY-EM3200”, “ZEISS”, and “SU8010” (see S3 Table for a complete list). For 11,314 of 174,046 articles articles with SEM related images, we were able to match the OCR output to one of the brandmarks in our set. We next downloaded the full text of the 11,314 articles identified in the previous step. For each article’s full text, we extracted portions of the text around occurrences of SEM brands (see S4 Table for a complete list of patterns). After completing this automated pipeline (Fig 3), R.A.K.R and J.M. manually compared the brandmark and model extracted from the OCR of the images to the brandmark and model listed in the full text of the article identified in the text snippets. We chose to perform manual labelling because we found it to be a more straightforward approach than implementing an automated pipeline. We found that, after some practice, we could execute this task at a rate of 5–10 seconds per article. PPT PowerPoint slide

PNG larger image

TIFF original image Download: Fig 3. Flowchart of implemented semi-automated pipeline. https://doi.org/10.1371/journal.pone.0326754.g003 We determined for each article whether the SEM used was declared in the full text. If this was the case, we also determined whether the brandmark of the SEM declared in the full text correctly matched the metadata of the instrument extracted from the images. If this was the case, we also determined whether the model of the SEM declared in the full text correctly matched the model of the instrument extracted from the images. In order to check for false positives introduced by our pipeline and to determine whether there could be a good explanation for the misidentification of SEMs in the detected articles, we selected a random sample of 150 articles from the set of detected articles labeled as problematic by our pipeline. S.S.H. reviewed each of the 150 articles (full text, figures and supplementary material) for mislabeling on our part and possible justifications for what would appear to be instrument misidentification. To assess whether articles with SEM misidentification were more likely to feature other errors, we looked for a specific common error in the estimation of the optical band gap from Tauc plots (Fig 6). We searched the full text of all 11,314 articles with identifiable SEM brandmarks for the string ‘tauc’. Our search returned 644 articles. 345 of 356 articles for which we had detected no errors in SEM identification included Tauc plots; 123 of 128 articles for which SEM was not declared in text included Tauc plots; and 154 of 160 articles for which the SEM instrument was incorrectly declared in the text included Tauc plots. To complete our analysis, we randomly sampled an additional 300 articles out of the 167,732 with SEM images but no identifiable SEM bandmarks, 283 of which included Tauc plots. We did not assess articles without SEM images since we did not download their full texts. We manually screened the main text and supplementary materials of every sampled article, annotating whether they had Tauc plots and flagging them if there was at least one Tauc plot where the band gap was improperly estimated because of the error described in Fig 6). Note that we did not assess for other errors (such as if the linear fit was not applied to the linear portion of the curve). We reproduce our annotations for the collected 944 articles in S6 Table. In order to estimate the scientific impact of the identified articles with SEM misidentification, we downloaded number of citations to each article in Scopus [25] using pybliometrics v3.5.2 [26] on January 29, 2024. We found Scopus records for 11,304 of 11,314 articles with SEM brandmarks and determined the presence of PubPeer comments on each article using a Feb 1, 2024 snapshot of PubPeer. Finally, in order to estimate characteristics of the authors of the identified articles with SEM misidentification, we downloaded author names and affiliations as reported on the article page and obtained country of of authorship from the affiliation strings. We did not, however, attempt automated author disambiguation. Code is available at github.com/amarallab/sem_images.

Limitations Our analysis is not without limitations. First, the size of image files for figures across the scientific literature increased substantially over the time period surveyed (Fig 2a and S2 Fig). As a result, brandmarks are likely more readily identifiable via OCR in more recent articles than in older ones. Thus, we cannot conclusively determine whether the problem has become worse in recent years. Similarly, image sizes varied considerably across journals and publishers, which may mislead comparative analysis of SEM misidentification rates. Second, we do not extract any text or figures from articles’ supplementary material, where there could be additional SEM images or reporting of SEM equipment. We arrived at the decision to exclude supplementary materials because journals and publisher application programming interfaces (APIs) did not make supplementary material available in a standardized format, unlike the main text and main text figures. Processing information from supplementary materials could recover more instances of SEM misidentification and might reclassify some cases where we found no identification of SEM instruments. However, manual inspection of articles (including inspection of supplementary materials) labeled as having SEM misidentification only yielded additional context in a minority of cases (Fig 5a). Thus, inclusion of supplementary materials is unlikely to alter false discovery rates. Third, because different journals, editors, reviewers, institutions and authors may hold distinct preferences concerning the inclusion of metadata banners in published articles, some journals and author nationalities may be artificially over- or under-represented among articles with SEM misidentification. Fourth, SEM misidentification could be much more prevalent than what we report. While our manual evaluation of a subset of articles with potential SEM misidentification demonstrates that our semi-automated pipeline has a low false discovery rate (Fig 5a), our analysis of existing observations of SEM misidentification made on PubPeer (Fig 5c) indicates that our approach has a high false negative rate. Based on this sensitivity, SEM misidentification could be more than twice as frequent as what we detect with our pipeline. Improvements on this method to improve sensitivity might involve using a lightweight convolutional neural network, vision transformer or other vision model to identify figures containing SEM images with metadata banners, which could then be manually inspected and annotated instead of relying on OCR to discern brandmarks.

Discussion We systematically investigated the misidentification of SEM instruments in more than a million articles from across 50 journals published by 4 different publishers. We found that in 24.7% of 11,314 articles for which instrument metadata could be extracted using our approach, the authors did not report the manufacturer and model of the instrument used and that for 21.2% of the articles, the image metadata did not match the SEM manufacturer or model listed in the text of the manuscript. We also find that articles with SEM misidentification are more likely to have existing observations of improprieties made on post-publication peer review site PubPeer than other MSE articles and that a subset of these articles within the subfield of electrochemistry are more likely to incorrectly estimate the optical band gap if the article features SEM misidentification. Although this analysis is correlative in nature, these findings suggests that SEM misidentification may be a tractable signature for flagging problematic MSE articles. Despite the limitations discussed in the previous section, our study reveals several facts that warrant action from all members of the MSE research community. The presence of repeated authorship (Fig 7b), shared image watermarks (S7 Table) and shared textual artifacts (S8 Table) strongly suggests that mass-production and submission of fabricated manuscripts is responsible for many cases of instrument misidentification. These text and image artifacts could serve as indicators of potential paper mill provenance, alongside other artifacts like tortured phrases [31] and shared images [14]. Indeed, some of the journals considered in our analysis have been advertised online as available venues for guaranteed acceptance (S5 Fig). While there may be no consensus in the field concerning the inclusion of metadata in images, either in supplementary materials or the main text, it is clear that by failing to request such information one makes it impossible to determine whether experimental instrumentation is misreported. For this reason, availability of metadata has been recognized as essential for ensuring image integrity and adherence to FAIR principles (findability, accessibility, interoperability and reusability) [32–36]. A more straightforward matter may be the establishment of more stringent standards for image resolution in published research (Fig 2 and S2 Fig). Low resolution images make it more difficult to identify reporting inconsistencies and image manipulation. Indeed, low standards for image resolution are likely to challenge recent efforts to automatically screen published and unpublished manuscripts for duplicated images [16, 37].

Acknowledgments We thank Dr. Justin Notestein for his helpful feedback, Dr. Thomas Stoeger for assisting us with accessing OpenAlex data and Helio Tejedor Navarro for technical support.

[END]
---
[1] Url: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0326754

Published and (C) by PLOS One
Content appears here under this condition or license: Creative Commons - Attribution BY 4.0.

via Magical.Fish Gopher News Feeds:
gopher://magical.fish/1/feeds/news/plosone/