https://news.mit.edu/2022/study-finds-wikipedia-influences-judicial-behavior-0727 Skip to content | Massachusetts Institute of Technology MIT Top Menu| * Education * Research * Innovation * Admissions + Aid * Campus Life * News * Alumni * About MIT * More | Search MIT Search websites, locations, and people [ ] See More Results Suggestions or feedback? MIT News | Massachusetts Institute of Technology Subscribe to MIT News newsletter Browse Enter keywords to search for news articles: [ ] Submit Browse By Topics View All - Explore: * Machine learning * Social justice * Startups * Black holes * Classes and programs Departments View All - Explore: * Aeronautics and Astronautics * Brain and Cognitive Sciences * Architecture * Political Science * Mechanical Engineering Centers, Labs, & Programs View All - Explore: * Abdul Latif Jameel Poverty Action Lab (J-PAL) * Picower Institute for Learning and Memory * Media Lab * Lincoln Laboratory Schools * School of Architecture + Planning * School of Engineering * School of Humanities, Arts, and Social Sciences * Sloan School of Management * School of Science * MIT Schwarzman College of Computing View all news coverage of MIT in the media - Subscribe to MIT newsletter - Close Breadcrumb 1. MIT News 2. Study finds Wikipedia influences judicial behavior Study finds Wikipedia influences judicial behavior Using a randomized field experiment, researchers discover that Wikipedia articles affect judges' legal reasoning. Rachel Gordon | MIT CSAIL Publication Date: July 27, 2022 Press Inquiries Press Contact: Rachel Gordon Email: rachelg@csail.mit.edu Phone: 617-258-0675 MIT Computer Science and Artificial Intelligence Laboratory Close Black and white illustration features a silhouetted person holding a gavel, who stands in the shadow of a large legal scale, one side of which is weighed down by a W representing the Wikipedia logo Caption: Using a randomized field experiment, researchers found that Wikipedia articles on decided cases, written by law students, guide both the decisions that judges cite as precedents and the textual content of their written opinions. Credits: Image: Alex Shipps/MIT CSAIL Previous image Next image Mixed appraisals of one of the internet's major resources, Wikipedia, are reflected in the slightly dystopian article "List of Wikipedia Scandals." Yet billions of users routinely flock to the online, anonymously editable, encyclopedic knowledge bank for just about everything. How this unauthoritative source influences our discourse and decisions is hard to reliably trace. But a new study attempts to measure how knowledge gleaned from Wikipedia may play out in one specific realm: the courts. A team of researchers led by Neil Thompson, a research scientist at MIT's Computer Science and Artificial Intelligence Laboratory (CSAIL), recently came up with a friendly experiment: creating new legal Wikipedia articles to examine how they affect the legal decisions of judges. They set off by developing over 150 new Wikipedia articles on Irish Supreme Court decisions, written by law students. Half of these were randomly chosen to be uploaded online, where they could be used by judges, clerks, lawyers, and so on -- the "treatment" group. The other half were kept offline, and this second group of cases provided the counterfactual basis of what would happen to a case absent a Wikipedia article about it (the "control"). They then looked at two measures: whether the cases were more likely to be cited as precedents by subsequent judicial decisions, and whether the argumentation in court judgments echoed the linguistic content of the new Wikipedia pages. It turned out the published articles tipped the scales: Getting a public Wikipedia article increased a case's citations by more than 20 percent. The increase was statistically significant, and the effect was particularly strong for cases that supported the argument the citing judge was making in their decision (but not the converse). Unsurprisingly, the increase was bigger for citations by lower courts -- the High Court -- and mostly absent for citations by appellate courts -- the Supreme Court and Court of Appeal. The researchers suspect this is showing that Wikipedia is used more by judges or clerks who have a heavier workload, for whom the convenience of Wikipedia offers a greater attraction. "To our knowledge, this is the first randomized field experiment that investigates the influence of legal sources on judicial behavior. And because randomized experiments are the gold standard for this type of research, we know the effect we are seeing is causation, not just correlation," says Thompson, the lead author of the study. "The fact that we wrote up all these cases, but the only ones that ended up on Wikipedia were those that won the proverbial 'coin flip,' allows us to show that Wikipedia is influencing both what judges cite and how they write up their decisions." "Our results also highlight an important public policy issue," Thompson adds. "With a source that is as widely used as Wikipedia, we want to make sure we are building institutions to ensure that the information is of the highest quality. The finding that judges or their staffs are using Wikipedia is a much bigger worry if the information they find there isn't reliable." A paper describing the study is being published in "The Cambridge Handbook of Experimental Jurisprudence" (Cambridge University Press, 2022). Joining Thompson on the paper are Brian Flannigan, Edana Richardson, and Brian McKenzie of Maynooth University in Ireland and Xueyun Luo of Cornell University. The researchers' statistical model essentially compared how much citation behavior changed for the treatment group (first difference: before versus after) and how that compared with the change that happened for the control group (second difference: treatment versus control). In 2018, Thompson first visited the idea of proving the causal role that Wikipedia plays in shaping knowledge and behavior by looking at how it shapes academic science. It turns out that adding scientific articles, in this case about chemistry, changed how the topic was discussed in scientific literature, and science articles added as references to Wikipedia received more academic citations as well. That led Brian McKenzie, an associate professor at Maynooth University, to make a call. "I was working with students to add articles to Wikipedia at the time I read Neil's research on the influence of Wikipedia on scientific research," explains McKenzie. "There were only a handful of Irish Supreme Court cases on Wikipedia so I reached out to Neil to ask if he wanted to design another iteration of his experiment using court cases." The Irish legal system proved the perfect test bed, as it shares a key similarity with other national legal systems such as the United Kingdom and United States -- it operates within a hierarchical court structure where decisions of higher courts subsequently bind lower courts. Also, there are relatively few Wikipedia articles on Irish Supreme Court decisions compared to those of the U.S. Supreme Court -- over the course of their project, the researchers increased the number of such articles tenfold. In addition to looking at the case citations made in the decisions, the team also analyzed the language used in the written decision using natural language processing. What they found were the linguistic fingerprints of the Wikipedia articles that they'd created. So what might this influence look like? Suppose A sues B in federal district court. A argues that B is liable for breach of contract; B acknowledges A's account of the facts but maintains that they gave rise to no contract between them. The assigned judge, conscious of the heavy work already delegated to his clerks, decides to conduct her own research. On reviewing the parties' submissions, the judge forms the preliminary view that a contract has not truly been formed and that she should give judgment for the defendant. To write his official opinion, the judge googles some previous decisions cited in B's brief that seem similar to the case between A and B. On confirming their similarity by reading the relevant case summaries on Wikipedia, the judge paraphrases some of the text of the Wikipedia entries in his draft opinion to complete his analysis. The judge then enters his judgment and publishes his opinion. "The text of a court's judgment itself will guide the law as it becomes a source of precedent for subsequent judicial decision-making. Future lawyers and judges will look back at that written judgment, and use it to decide what its implications are so that they can treat 'like' cases alike," says coauthor Brian Flanagan. "If the text itself is influenced, as this experiment shows, by anonymously sourced internet content, that's a problem. For the many potential cracks that have opened up in our "information superhighway" that is the internet, you can imagine that this vulnerability could potentially lead to adversarial actors manipulating information. If easily accessible analysis of legal questions is already being relied on, it behooves the legal community to accelerate efforts to ensure that such analysis is both comprehensive and expert." Share this news article on: * Twitter * Facebook * LinkedIn * Reddit * Print Paper Paper: "Trial by Internet: A Randomized Field Experiment on Wikipedia's Influence on Judges' Legal Reasoning" Press Mentions Popular Science Researchers from CSAIL, Cornell University, and Maynooth University have released a study concluding that judges in Ireland are utilizing Wikipedia articles to help inform their decisions, reports Colleen Hagerty for Popular Science. Based on their findings, the researchers suggest "the legal community increases its efforts to monitor and fact-check legal information posted on Wikipedia." Full story via Popular Science - Independent Researchers from CSAIL and elsewhere have found that Irish judges are using Wikipedia articles as a source in their rulings, reports Shane Phelan for Independent. "This work shows that Wikipedia reaches even farther than that, into high-stakes, formalized processes like legal judgments," says research scientist Neil Thompson. "The worst outcome would be for a judge's reliance on Wikipedia to lead them to decide a case differently than they would have if they had read either an expert secondary source or the cited precedent itself." Full story via Independent - Previous item Next item Related Links * Neil Thompson * Computer Science and Artficial Intelligence Laboratory (CSAIL) * MIT Schwarzman College of Computing * MIT Sloan School of Management Related Topics * Research * Law * Technology and society * Internet * Computer science and technology * Natural language processing * Computer Science and Artificial Intelligence Laboratory (CSAIL) * MIT Schwarzman College of Computing * MIT Sloan School of Management Related Articles MIT researchers have created an automated text-generating system that pinpoints and replaces specific information in relevant Wikipedia sentences, while keeping the language similar to how humans write and edit. Automated system can rewrite outdated sentences in Wikipedia articles MIT Professor Regina Barzilay (left) and CSAIL PhD student Tal Schuster are studying detectors of machine-generated text. Better fact-checking for fake news [placeholder--frontpage--featured-news] Peering under the hood of fake-news detectors Previous item Next item More MIT News Danna Freedman For Danna Freedman, an impasse is an invitation The chemistry professor embraces the most challenging moments of her work to design molecules for quantum information science. Read full story - adhesive patch that produces ultrasound images MIT engineers develop stickers that can see inside the body New stamp-sized ultrasound adhesives produce clear images of heart, lungs, and other internal organs. Read full story - analog deep learning processor New hardware offers faster computation for artificial intelligence, with much less energy Engineers working on "analog deep learning" have found a way to propel protons through solids at unprecedented speeds. Read full story - Photo of MIT buildings centered on 314 Main Street, a 17-story glass building. A courtyard with grass, tables, chairs, and walkways is in the foreground. 3 Questions: John Durant on the new MIT Museum at Kendall Square The MIT Museum director describes how the museum is reinventing itself for the 21st century Read full story - A man in a suit runs along a sidewalk with his arm raised, signaling to a red bus stopped along a curb. Sprint then stop? The brain is wired for the math to make it happen To ensure a quick halt, brain circuit architecture avoids a slow process of integration in favor of quicker differentiation, study finds. Read full story - passenger with mask on in airplane Friendly skies? Study charts Covid-19 odds for plane flights Researchers calculate chances of catching the illness when aloft, though pandemic conditions keep shifting. Read full story - * More news on MIT News homepage - More about MIT News at Massachusetts Institute of Technology This website is managed by the MIT News Office, part of the MIT Office of Communications. News by Schools/College: * School of Architecture and Planning * School of Engineering * School of Humanities, Arts, and Social Sciences * MIT Sloan School of Management * School of Science * MIT Schwarzman College of Computing Resources: * About the MIT News Office * MIT News Press Center * Terms of Use * Press Inquiries * Filming Guidelines * RSS Feeds Tools: * Subscribe to MIT Daily/Weekly * Subscribe to press releases * Submit campus news Massachusetts Institute of Technology MIT Top Level Links: * Education * Research * Innovation * Admissions + Aid * Campus Life * News * Alumni * About MIT * Join us in building a better world. Massachusetts Institute of Technology 77 Massachusetts Avenue, Cambridge, MA, USA Recommended Links: * Visit * Map (opens in new window) * Events (opens in new window) * People (opens in new window) * Careers (opens in new window) * Contact * Privacy * Accessibility * + Social Media Hub + MIT on Twitter + MIT on Facebook + MIT on YouTube + MIT on Instagram