https://www.nature.com/articles/s41591-021-01535-y Skip to main content Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript. Advertisement Advertisement Nature Medicine * View all journals * Search * My Account Login * Explore content * About the journal * Publish with us * Sign up for alerts * RSS feed 1. nature 2. nature medicine 3. correspondence 4. article The lesson of ivermectin: meta-analyses based on summary data alone are inherently unreliable Download PDF * Correspondence * Published: 22 September 2021 The lesson of ivermectin: meta-analyses based on summary data alone are inherently unreliable * Jack M. Lawrence^1, * Gideon Meyerowitz-Katz ORCID: orcid.org/0000-0002-1156-9345^2, * James A. J. Heathers^3, * Nicholas J. L. Brown^4 & * Kyle A. Sheldrick ORCID: orcid.org/0000-0002-2198-8025^5 Nature Medicine (2021)Cite this article * 696 Altmetric * Metrics details Subjects * Medical research * Randomized controlled trials Download PDF To the Editor -- The global demand for prophylactic and treatment options for COVID-19 has in turn created a demand for both randomized clinical trials, and the synthesis of those trials into meta-analyses by systematic review. This process has been fraught, and has demonstrated the inherent risks in current approaches and accepted standards of quantitative evidence synthesis when dealing with high volumes of recent, often unpublished trial data of variable quality. Research into the use of ivermectin (a drug that has an established safety and efficacy record in many parasitic diseases) for the treatment and/or prophylaxis of COVID-19 has illustrated this problem well. Recently, we described flaws in one randomized control trial of ivermectin^1, the results of which represented more than 10% of the overall effect in at least two major meta-analyses^2,3. We described several irregularities in the data that could not be consistent with them being experimentally derived^4. That study has now been withdrawn by the preprint server^5 on which it was hosted. We also raised concerns about unexpected stratification across baseline variables in another randomized controlled trial for ivermectin^6, which were highly suggestive of randomization failure. We have requested data from the authors but, as of 6 September 2021, have not yet received a response. This second ivermectin study has now been published^6, and there is still no response from the authors in a request for data. The authors of one recently published meta-analysis of ivermectin for COVID-19^3 have publicly stated that they will now reanalyze and republish their now-retracted meta-analysis and will no longer include either of the two papers just mentioned. As these two papers^ 1,6 were the only studies included in that meta-analysis to demonstrate an independently significant reduction in mortality, the revision will probably show no mortality benefit for ivermectin. Several other studies that claim a clinical benefit for ivermectin are similarly fraught, and contain impossible numbers in their results, unexplainable mismatches between trial registry updates and published patient demographics, purported timelines that are not consistent with the veracity of the data collection, and substantial methodological weaknesses. We expect further studies supporting ivermectin to be withdrawn over the coming months. Since the above primary studies were published, many hundreds of thousands of patients^7 have been dosed with ivermectin, relying on an evidence base that has substantially evaporated under close scrutiny. Relying on low-quality or questionable studies in the current global climate presents severe and immediate harms. The enormous impact of COVID-19 and the consequent urgent need to demonstrate the clinical efficacy of new therapeutic options provides fertile ground for even poorly evidenced claims of efficacy to be amplified, both in the scientific literature and on social media. This context can lead to the rapid translation of almost any apparently favorable conclusion from a relatively weak trial or set of trials into widespread clinical practice and public policy. We believe that this situation requires immediate remediation. The most salient change required is a change in perspective on the part of both primary researchers and those who bring together the results of individual studies to draw wider conclusions. Specifically, we propose that clinical research should be seen as a contribution of data toward a larger omnibus question rather than an assemblage of summary statistics. Most, if not all, of the flaws described above would have been immediately detected if meta-analyses were performed on an individual patient data (IPD) basis. In particular, irregularities such as extreme terminal digit bias and the duplication of blocks of patient records would have been both obvious and immediately interrogable from raw data if provided. We recommend that meta-analysts who study interventions for COVID-19 should request and personally review IPD in all cases, even if IPD synthesis techniques are not used. In a similar vein, all clinical trials published on COVID-19 should immediately follow best-practice guidelines and upload anonymized IPD so that this type of analysis can occur. Any study for which authors are not able or not willing to provide suitably anonymized IPD should be considered at high risk of bias for incomplete reporting and/or excluded entirely from meta-syntheses. Hurdles to the release of IPD from clinical trials are well described, and generally addressable with careful anonymization and integration of data sharing plans at the ethical approval stage of trial planning. We recognize that this is a change to long-accepted practice and is substantially more rigorous than the standards that are typically currently applied, but we believe that what has happened in the case of ivermectin justifies our proposal: a poorly scrutinized evidence base supported the administration of millions of doses of a potentially ineffective drug globally, and yet when this evidence was subjected to a very basic numerical scrutiny it collapsed in a matter of weeks. This research has created undue confidence in the use of ivermectin as a prophylactic or treatment for COVID-19, has usurped other research agendas, and probably resulted in inappropriate treatment or substandard care of patients. We recognize that by recommending IPD review by default for meta-analysis of potential therapeutic agents in COVID-19 we are calling for change to nearly universally accepted practice over many decades, but the consequent potential for patient harm on a global scale demands nothing less. References 1. 1. Elgazzar, A. et al. Preprint at https://www.researchsquare.com/ article/rs-100956/v3 (2020). 2. 2. Bryant, A. et al. Am. J. Ther. 28, e434-e460 (2021). Article Google Scholar 3. 3. Hill, A. et al. Open Forum Infectious Diseases https://doi.org/ 10.1093/ofid/ofab358 (2021). 4. 4. Davey, M. The Guardian https://www.theguardian.com/science/2021/ jul/16/ huge-study-supporting-ivermectin-as-covid-treatment-withdrawn-over-ethical-concerns (16 July 2021). 5. 5. Elgazzar, A. et al. Preprint at https://doi.org/10.21203/ rs.3.rs-100956/v2 (2020). 6. 6. Shakhsi Niaee, M. et al. Asian Pac. J. Trop. Med. 14, 266-273 (2021). Article Google Scholar 7. 7. Mega, E. R. Nature 586, 481-482 (2020). Article Google Scholar Download references Author information Affiliations 1. St George's, University of London, London, UK Jack M. Lawrence 2. School of Health and Society, University of Wollongong, New South Wales, Australia Gideon Meyerowitz-Katz 3. Cipher Skin, Denver, CO, USA James A. J. Heathers 4. Linnaeus University, Vaxjo, Sweden Nicholas J. L. Brown 5. University of New South Wales, Kensington, Australia Kyle A. Sheldrick Authors 1. Jack M. Lawrence View author publications You can also search for this author in PubMed Google Scholar 2. Gideon Meyerowitz-Katz View author publications You can also search for this author in PubMed Google Scholar 3. James A. J. Heathers View author publications You can also search for this author in PubMed Google Scholar 4. Nicholas J. L. Brown View author publications You can also search for this author in PubMed Google Scholar 5. Kyle A. Sheldrick View author publications You can also search for this author in PubMed Google Scholar Contributions All authors contributed equally to the writing of this manuscript. K.S. and G.M.K. primarily contacted authors requesting IPD for randomized trials. Corresponding author Correspondence to Kyle A. Sheldrick. Ethics declarations Competing interests The authors declare no competing interests. Rights and permissions Reprints and Permissions About this article Verify currency and authenticity via CrossMark Cite this article Lawrence, J.M., Meyerowitz-Katz, G., Heathers, J.A.J. et al. The lesson of ivermectin: meta-analyses based on summary data alone are inherently unreliable. Nat Med (2021). https://doi.org/10.1038/ s41591-021-01535-y Download citation * Published: 22 September 2021 * DOI: https://doi.org/10.1038/s41591-021-01535-y Share this article Anyone you share the following link with will be able to read this content: Get shareable link Sorry, a shareable link is not currently available for this article. Copy to clipboard Provided by the Springer Nature SharedIt content-sharing initiative Download PDF Advertisement Advertisement Explore content * Research articles * Reviews & Analysis * News & Comment * Podcasts * Current issue * Collections * Follow us on Facebook * Follow us on Twitter * Sign up for alerts * RSS feed About the journal * Aims & Scope * Journal Information * Our publishing models * Content Types * Clinical Research * About the Editors * Web Feeds * Nature Medicine Classic Collection * Editorial Values Statement * Journal Impact * Contact * Posters * Editorial policies Publish with us * For Reviewers * For Authors * Submit manuscript Search Search articles by subject, keyword or author [ ] Show results from [All journals] Search Advanced search Quick links * Explore articles by subject * Find a job * Guide to authors * Editorial policies Nature Medicine (Nat Med) ISSN 1546-170X (online) ISSN 1078-8956 (print) nature.com sitemap Nature portfolio * About us * Press releases * Press office * Contact us * * * Discover content * Journals A-Z * Articles by subject * Nano * Protocol Exchange * Nature Index Publishing policies * Nature portfolio policies * Open access Author & Researcher services * Reprints & permissions * Research data * Language editing * Scientific editing * Nature Masterclasses * Nature Research Academies Libraries & institutions * Librarian service & tools * Librarian portal * Open research * Recommend to library Advertising & partnerships * Advertising * Partnerships & Services * Media kits * Branded content Career development * Nature Careers * Nature Conferences * Nature events Regional websites * Nature Africa * Nature China * Nature India * Nature Italy * Nature Japan * Nature Korea * Nature Middle East Legal & Privacy * Privacy Policy * Use of cookies * Manage cookies/Do not sell my data * Legal notice * Accessibility statement * Terms & Conditions * California Privacy Statement Springer Nature (c) 2021 Springer Nature Limited Close Nature Briefing Sign up for the Nature Briefing newsletter -- what matters in science, free to your inbox daily. Email address [ ] Sign up [ ] I agree my information will be processed in accordance with the Nature and Springer Nature Limited Privacy Policy. Close Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing * *