https://arstechnica.com/tech-policy/2023/05/federal-judge-no-ai-in-my-courtroom-unless-a-human-verifies-its-accuracy/ Skip to main content * Biz & IT * Tech * Science * Policy * Cars * Gaming & Culture * Store * Forums Subscribe [ ] Close Navigate * Store * Subscribe * Videos * Features * Reviews * RSS Feeds * Mobile Site * About Ars * Staff Directory * Contact Us * Advertise with Ars * Reprints Filter by topic * Biz & IT * Tech * Science * Policy * Cars * Gaming & Culture * Store * Forums Settings Front page layout Grid List Site theme light dark Sign in Trust in humans -- Federal judge: No AI in my courtroom unless a human verifies its accuracy Judge wary of AI "hallucinations," says it isn't acceptable for legal briefing. Jon Brodkin - May 31, 2023 7:30 pm UTC Illustration of a judge's gavel on a digital background resembling a computer circuit board. Enlarge Getty Images | the-lightwriter reader comments 113 with A federal judge in Texas has a new rule for lawyers in his courtroom: No submissions written by artificial intelligence unless the AI's output is checked by a human. US District Judge Brantley Starr also ordered lawyers to file certificates attesting that their filings are either written or reviewed by humans. "All attorneys appearing before the Court must file on the docket a certificate attesting either that no portion of the filing was drafted by generative artificial intelligence (such as ChatGPT, Harvey.AI, or Google Bard) or that any language drafted by generative artificial intelligence was checked for accuracy, using print reporters or traditional legal databases, by a human being," according to a new "judge-specific requirement" in Starr's courtroom. A certification must be submitted for each case and would cover all filings in the case. A sample certification says the requirements apply to any language in a filing, "including quotations, citations, paraphrased assertions, and legal analysis." Starr, a Trump nominee in US District Court for the Northern District of Texas, said that AI "platforms are incredibly powerful and have many uses in the law: form divorces, discovery requests, suggested errors in documents, anticipated questions at oral argument. But legal briefing is not one of them." AI "platforms in their current states are prone to hallucinations and bias. On hallucinations, they make stuff up--even quotes and citations." Starr's new standing order on AI was posted yesterday, according to an article by legal expert Eugene Volokh. "Note that federal judges routinely have their own standing orders for lawyers practicing in their courtrooms. These are in addition to the local district rules, and to the normal Federal Rules of Civil and Criminal Procedure," Volokh wrote. Advertisement Lawyer cited fake cases invented by ChatGPT Starr's order came after New York-based lawyer Steven Schwartz admitted that he used ChatGPT to help write court filings that cited six nonexistent cases invented by the artificial intelligence tool. Schwartz and his associates are awaiting possible punishments from Judge Kevin Castel of US District Court for the Southern District of New York. In what Castel called "an unprecedented circumstance," the judge said filings from the plaintiff's lawyers included six "bogus judicial decisions with bogus quotes and bogus internal citations." The filings included names of made-up cases and a series of "excerpts" from bogus rulings that cited additional fake precedents invented by AI. An affidavit filed by Schwartz said he "greatly regrets having utilized generative artificial intelligence to supplement the legal research performed herein and will never do so in the future without absolute verification of its authenticity." He also said he had "never utilized ChatGPT as a source for conducting legal research prior to this occurrence and therefore was unaware of the possibility that its content could be false." Judge: AI swears no oath, holds no allegiance Starr's new standing order discussed potential biases in AI tools. "While attorneys swear an oath to set aside their personal prejudices, biases, and beliefs to faithfully uphold the law and represent their clients, generative artificial intelligence is the product of programming devised by humans who did not have to swear such an oath," he wrote. AI "systems hold no allegiance to any client, the rule of law, or the laws and Constitution of the United States (or, as addressed above, the truth)," Starr continued. "Unbound by any sense of duty, honor, or justice, such programs act according to computer code rather than conviction, based on programming rather than principle. Any party believing a platform has the requisite accuracy and reliability for legal briefing may move for leave and explain why." Going forward, Starr's order said the court "will strike any filing from an attorney who fails to file a certificate on the docket attesting that the attorney has read the Court's judge-specific requirements and understands that he or she will be held responsible under Rule 11 for the contents of any filing that he or she signs and submits to the Court, regardless of whether generative artificial intelligence drafted any portion of that filing." reader comments 113 with Jon Brodkin Jon has been a reporter for Ars Technica since 2011 and covers a wide array of telecom and tech policy topics. Jon graduated from Boston University with a degree in journalism and has been a full-time journalist for over 20 years. Advertisement Channel Ars Technica - Previous story Next story - Related Stories Today on Ars * Store * Subscribe * About Us * RSS Feeds * View Mobile Site * Contact Us * Staff * Advertise with us * Reprints Newsletter Signup Join the Ars Orbital Transmission mailing list to get weekly updates delivered to your inbox. Sign me up - CNMN Collection WIRED Media Group (c) 2023 Conde Nast. All rights reserved. Use of and/or registration on any portion of this site constitutes acceptance of our User Agreement (updated 1/1/20) and Privacy Policy and Cookie Statement (updated 1/1 /20) and Ars Technica Addendum (effective 8/21/2018). Ars may earn compensation on sales from links on this site. Read our affiliate link policy. Your California Privacy Rights | Do Not Sell My Personal Information The material on this site may not be reproduced, distributed, transmitted, cached or otherwise used, except with the prior written permission of Conde Nast. Ad Choices