https://www.cebm.ox.ac.uk/news/views/the-prosecutors-fallacy Cookies on this website We use cookies to ensure that we give you the best experience on our website. If you click 'Accept all cookies' we'll assume that you are happy to receive all cookies and you won't see this message again. If you click 'Reject all non-essential cookies' only necessary cookies providing core functionality such as security, network management, and accessibility will be enabled. Click 'Find out more' for information on how to change your cookie settings. Accept all cookies Reject all non-essential cookies Nuffield Department of Primary Care Health Sciences, University of Oxford * About * Research * Study with us * Team * Public involvement * News * Events Centre for Evidence-Based Medicine (CEBM), University of Oxford University of Oxford * Site Map * Contact * Cookies * Log in * Home * About us + About us + Who we are + Our history + Benefactors and patrons + David Sackett Fellowship 2022 + Doug Altman Scholarship + Contact us * Study with us + Study with us + Tutor expertise: research and supervision + EBHC Study Routes Overview + The Masters to DPhil journey + Oxford Medical School * Resources + Resources + EBM tools + COVID-19 Evidence Service + Levels of evidence + Data extraction tips for meta-analysis + Reports + Top tips * Research + About our research + Projects + Publications + Evidence-Based Health Care podcasts + Evidence Resonates: EBHC DPhil research * News & views + News and views + CEBM in the news + Podcasts * Events Search [ ] 1. 2. News & views 3. Views 4. The Prosecutor's Fallacy The Prosecutor's Fallacy Share Share Share Kathy Taylor 16 July 2018 Research reviews & expert opinions Getting the wrong end of the stick: The Prosecutor's Fallacy We may all be guilty of it at some time or other The probability that this nurse's shifts would coincide with so many deaths and resuscitations by chance is 1 in 342 million, so she must be guilty. Given your positive test result, the chance of your dying within 10 years is 99.9%. He mustn't love me anymore, as it's been 3 days and he hasn't returned my call. Kathy Taylor, medical statistician The three statements above have one thing in common - they are all examples of The Prosecutor's Fallacy (1). This is a logical error involving conditional probabilities - a measure of the chance, likelihood, or probability of X when Y has happened, Y being something that modifies the chance. The Prosecutor's Fallacy can be avoided by making sure the probability answers the right question, by focusing on how the evidence applies to the 'defendant' and not on the 'evidence' alone in the absence of other relevant factors. The Prosecutor's Fallacy is most often associated with miscarriages of justice. It's when the probability of innocence given the evidence is wrongly assumed to equal an infinitesimally small probability that that evidence would occur if the defendant was innocent. Consequently, highly improbable innocent explanations have led to the assumption of guilt, as with the murder convictions of Sally Clark in 1999 and Lucia de Berk in 2003 (2). Leonard Mlodinow's doctor made the same logical error when he gave an alarming prognosis back in 1989 (3). His doctor had misinterpreted the 1 in 1000 probability that the HIV test will produce a positive result when the blood is not infected (the false positive rate), interpreting it as the probability that his blood was not infected if it gave a positive test result. Only 1 in 10,000 people from a low-risk population who are tested is eventually confirmed as being infected, but because the false positive rate is 1 in 1000, 10 people in 10,000 will test positive who aren't infected, compared with 1 who is. Thus, the odds that Mlodinow was infected was 10 to 1 against. The fact that he was from a low-risk population suggested that he was unlikely to be infected. The third example above ("He mustn't love me any more...") reflects someone wrongly assuming that, given a delayed response to their message, the probability that they have lost favour is equal to the high probability of a delayed response that would occur if they had lost favour, whereas there is higher probability that there is some other reason for the delay. Variants of this example, which lead to misunderstandings between friends, families or colleagues make it the most common example of the Prosecutor's Fallacy. In these examples, the conditional probabilities are inverted, but doing this ignores both the alternative explanations and the associated probability of "guilt" (disease or lost favour) before the new evidence (the result of a test or delayed response) occurs. In the courtroom, this assumed (or prior) probability is the probability of guilt or innocence based on all the other evidence, and in the medical context, it's the prevalence of the disease. Ignoring the base rate is a common error (4). Bayes' theorem (5) shows how the two conditional probabilities are related in updating the prior probability following the addition of new information: KT1.png Therefore, the Prosecutors Fallacy is a subtle error that requires careful thought to understand, which makes teaching it all the more challenging. Visual explanations may be helpful. The difference between the two conditional probabilities can be illustrated by considering the results of a diagnostic test in one million people, at two levels of prevalence, and using blocks with areas scaled to represent the number of people (shown in black). The test has 98% sensitivity and a false positive rate of 1% (shown in red). KT2.png The first conditional probability is P(Positive test result|No disease), which is the false positive rate. It is the same for (a) and (b): KT3.png The numerator is the number of people who have no disease and have tested positive (9998 and 8000 for (a) and (b) respectively). The denominator is the total number with no disease irrespective of their test result (labelled No Disease). This probability does not vary according to the prevalence. The second conditional probability is P(No disease|Positive test result), i.e. the chance that you don't have the disease, even though the test is positive: KT4.png The numerator is the same, as before, but the denominator is the number of people in the whole sample with a positive test result (shaded area). This probability depends on the prevalence and also on the relative number of those with positive tests, with and without the disease. Notice that at 0.02% prevalence the two conditional probabilities differ by 97% but at 20% prevalence the difference is only 3%. Therefore, the Prosecutor's Fallacy is not an issue when the prevalence (or prior likelihood of guilt) is high, because the conditional probabilities are similar. A simpler version of the Prosecutor's Fallacy arises when a defendant shares the physical characteristics of the perpetrator of the crime, and the probability of innocence given this match - P(Innocent|Match) - is wrongly assumed to equal the infinitesimally small chance of a random person in the population sharing those characteristics - P (Match). These probabilities are not equal, as shown above by Bayes' theorem. Unlike the previous examples, conditional probabilities are not inverted, but like the previous examples, this logical error arises from answering the wrong question, by focusing purely on the evidence and not on the evidence as it relates to the defendant, including other factors that may modify the likelihoods. References 1. Thompson WC, Schumann EL. Interpretation of statistical evidence in criminal trials: The prosecutor's fallacy and defense attorney's fallacy. Law and Human Behaviour 1987; 11:167-187 2. Hubert L, Wainer H. A statistical guide for the ethically perplexed. CRP Press, Taylor & Francis Group. 2013. 3. Mlodinow L. The drunkard's walk: how randomness rules our lives. Allen Lane. 2008 4. Tversky, A.; Kahneman, D. (1974). "Judgment under Uncertainty: Heuristics and Biases". Science. 185(4157): 1124-1131. 5. Altman D. Practical statistics for medical research. Chapman & Hall. 1993 Kathy receives funding from the NHS National Institute for Health Research (NIHR) Programme for Applied Research. The views expressed are those of the author and not necessary of the NHS or the NIHR. Want to learn how we teach statistics and other key topics in Evidence-Based HealthCare? Then join us at our annual teaching course 14 - September 2020. More details here. --------------------------------------------------------------------- [nuffield_p] (c) 2023 University of Oxford. All blog posts and resources are published under a CC BY 4.0 license. * Views disclaimer * Blog moderation * Freedom of Information * Privacy Policy * Copyright Statement * Accessibility Statement * Site Map * Contact * Cookies * Log in * Home * About us + About us + Our history + Who we are + Benefactors and patrons + Consulting services + Contact us + David Sackett Fellowship 2022 + Doug Altman Scholarship * Study with us + Study with us + EBHC FAQs o What is the price difference between credit and non-credit bearing modules? o Are MSc applicants eligible for Research Council Funding? o What's the difference between the Annual Award Fee, the Module/Course Fee, and the Dissertation Fee? o Is accommodation included in the price of the courses? o Can a University Loan be used to fund the course fees? o Is the price of completing one of the fully online courses the same as the 'Oxford week' blended courses? o Are Award, Course and Dissertation fees the same every year? o How can I find out if this programme is a good fit for my specific research and career development interests? o What kind of project do people do for their MSc Dissertation? o Can a short courses completed 'For Credit', count towards a Masters award if enrolled at a later date? o Will I get a formal Oxford University Certificate for completing one of the short courses? o Can the programme be completed entirely online without attending Oxford? o Will I have an Oxford Email address for the duration of my studies? o How are Courses/Modules assessed? o How many contact hours are there in the face to face 'Oxford weeks'? o What kind of time commitment is required in order to undertake the dissertation element of the MSc programme? o What are the maximum and minimum number of years the MSc, PgCert, and PgDip programmes can be completed in? o What is the difference between completing a professional short course 'for credit' or 'not for credit'? o Where can I find the dates when all the modules/ short courses are running? o Can the MSc be completed in one year? o Does the mode of delivery still allow you to be able to work full time? o Is there a minimum or maximum number of modules required per year as part of the MSc? o Do modules/Short Courses run more than once a year? o What date do short-course applications close? o What is the process for applying for a short course or award? o What is the difference between 'Blended', 'Fully Online' and 'By Attendance' delivery modes? o Do you operate a 'waiting list' for the Short Courses? o Is a certain level of English proficiency required to apply for the programme and how does this have to be demonstrated? o Will an application for an MSc award still be considered if it does not meet the minimum requirement of a First Class or strong Upper Second Class Honours Degree? o Is a Healthcare background a requirement for completing the Awards or Short Courses? o How do I evidence the commitment of my employer to allow time for study, in my application? o Are all the Awards and short courses open to international students and is the price of the courses and modules the same? o Where can I find information about whether my international qualification and grades are equivalent to what is required for my application to be considered? o Can the focus of a DPhil thesis be based on a project outside of the UK? o How long does it take to complete the DPhil? o Is the part-time DPhil delivered through distance learning, or is attendance at the University required? o How are Supervisors selected and allocated for the DPhil and can the focus for potential projects be discussed prior to an application? o I have previously attended a module as a stand-alone student. Can I import this module if I join the MSc Evidence Based Health Care Programme? o For face-to-face modules with an intensive teaching week, how much additional study time is required after the week in Oxford and prior to the assignment submission? o Is it possible to complete two modules that are offered concurrently? o How can I get in touch with any current students or recent graduates? o Where can I find details of the specific topic areas covered by the module? o Is it possible to register interest in a course with a view to joining it in the future? o What is the average class size for your modules/short courses? o Are graduation ceremonies held at Oxford? Do all programmes on EBHC get a graduation ceremony? o What is the accommodation cost for modules with Oxford weeks likely to be? o Is there a difference in cost between taking the module for credit or not for credit? + Example student career paths o Secondary Care Clinician o Primary Care Clinician o Dentist o Veterinary Clinician o Healthcare Researcher o Educational role o Publishing & Communications o Policy o Systematic Reviewer + Oxford Medical School + EBHC DPhil Clusters + EBHC Study Routes Overview + MSc in Evidence-Based Health Care + MSc in EBHC Teaching and Education + MSc in EBHC Systematic Reviews + MSc in EBHC Medical Statistics + PGCert in Teaching Evidence-Based Health Care + PGCert in Qualitative Health Research Methods + PGCert in Health Research + Practice of Evidence-Based Health Care + Introduction to Study Design and Research Methods + Clinical Trial Management + Mixed Methods in Health Research + Knowledge into Action + Evidence-Based Diagnosis and Screening + Introduction to Statistics for Health Care Research + Systematic Reviews + Randomised Controlled Trials + Qualitative Research Methods + Essential Medical Statistics + The History and Philosophy of Evidence-Based Health Care + Teaching Evidence-Based Practice + Realist Reviews and Realist Evaluation + Complex Reviews + Meta-analysis + Statistics for Clinical Trials + Big Data Epidemiology + Developing Practice in Medical Education + Course Design, Assessment and Evaluation + Developing Online Education and Resources (online only) + Statistical Computing with R and Stata (online only) + Clinical Prediction Rules + Advanced Qualitative Research Methods + Postgraduate Diploma in Health Research + Health Behaviour Change + Qualitative and Mixed Methods Systematic Reviews + Fundamentals of Evidence Based Health Care Leadership + Economics of Health Care + The Masters to DPhil journey + Oxford Medical School o Oxford Medical School o Special Study Modules (SSM) o Graduate entry/accelerated medical degree o Academic Special Interest Projects (ASIP) o Medical Electives o Standard Entry Medical Course o Final Honours Scheme (FHS) o EBM Special Study Theme (SST) + Tutor expertise: research and supervision + Systematic reviews + Meta-analysis + Diagnostic tests + Qualitative research methods + Monitoring and management + Teaching and leadership + Evidence synthesis + Interventions + Social Prescribing + Diagnosis & Prognosis + Big data + Statistics + Epidemiology * Resources + Resources + EBM tools o Finding the evidence: a how-to guide o Critical Appraisal tools o Number Needed to Treat (NNT) o Likelihood Ratios o CATMaker o Study designs o Asking focused questions o SpPin and SnNout o Data Extraction in Meta-analysis o Making a decision o PaT Plot tool for randomised trials o Searching exercise 'warm-up' o Glossary o EBM in Schools + Levels of evidence o Levels of Evidence: An introduction o OCEBM Levels of Evidence o Oxford Centre for Evidence-Based Medicine: Levels of Evidence (March 2009) o Explanation of the 2011 OCEBM Levels of Evidence + Reports o Defining value-based healthcare in the NHS o Improving patient safety o Can gardens, libraries and museums improve wellbeing through social prescribing? o Update to the association between Oral Hormone Pregnancy Tests, including Primodos, and congenital anomalies + Top tips + Systematic Reviews + Medical Student Resources o CEBM Presentations o Graduate Entry Resources o Graduate Entry Resources + EBM Library + COVID-19 Evidence Service * Research + Our research vision, philosophy and methods + About our research + Projects o Research projects + Protocols o Hormone pregnancy test use in pregnancy and risk of abnormalities in the offspring: a systematic review protocol + Our response to COVID-19 + Tamiflu as a treatment for influenza o 2014 o 2015 o 2016 o 2017 o 2018 o 2019 + Anticoagulation care o 2006 o 2008 o 2010 o 2011 o 2012 o 2014 o 2015 o 2016 o 2017 o 2018 + The Primodos story o 2018 o 2019 + The harms of transvaginal mesh o 2017 o 2018 o 2019 o 2021 + Evidence Resonates: EBHC DPhil research + Teaching and Education + Electronic Cigarettes for Smoking Cessation: Cochrane Living Systematic Review + Electronic Cigarettes for Smoking Cessation: Cochrane Living Systematic Review: press coverage + E-Cigarette for Smoking Cessation Cochrane Systematic Review: meet the team + Transmission of SARS-CoV-2 + Trust the Evidence + Evidence Synthesis Working Group + Social Prescribing + Preventable Deaths * News & views + News and views + Podcasts + CEBM in the news + Views disclaimer + Blog moderation * Events