https://www.techdirt.com/2025/11/13/judge-orders-openai-to-give-lawyers-20-million-private-chats-thinks-anonymization-can-keep-them-private/ [ ] Techdirt. [ ] * Sign In * Register * Preferences Techdirt [ ] * TechDirt * GreenHouse * Free Speech * Error 402 * Ctrl-Alt-Speech * Deals * Jobs * Support Techdirt [podcast-ti] Daily Deal: MagStack Foldable 3-in-1 Wireless Charging Station with Floating Stand Furloughed Employees Sue Administration For Adding Partisan Wording To Their Out-Of-Office Messages Judge Orders OpenAI To Give Lawyers 20 Million Private Chats, Thinks 'Anonymization' Can Keep Them Private [privacy] Privacy from the seems-like-a-problem dept Thu, Nov 13th 2025 10:42am - Mike Masnick A federal magistrate judge just ordered that the private ChatGPT conversations of 20 million users be handed over to the lawyers for dozens of plaintiffs, including news organizations. Those 20 million people weren't asked. They weren't notified. They have no say in the matter. Last week, Magistrate Judge Ona Wang ordered OpenAI to turn over a sample of 20 million chat logs as part of the sprawling multidistrict litigation where publishers are suing AI companies--a mess of consolidated cases that kicked off with the NY Times' lawsuit against OpenAI. Judge Wang dismissed OpenAI's privacy concerns, apparently convinced that "anonymization" solves everything. Even if you hate OpenAI and everything it stands for, and hope that the news orgs bring it to its knees, this should scare you. A lot. OpenAI had pointed out to the judge a week earlier that this demands from the news orgs would represent a massive privacy violation for ChatGPT's users. News Plaintiffs demand that OpenAI hand over the entire 20M log sample "in readily searchable format" via a "hard drive or [] dedicated private cloud." ECF 656 at 3. That would include logs that are neither relevant nor responsive--indeed, News Plaintiffs concede that at least 99.99% of the logs are irrelevant to their claims. OpenAI has never agreed to such a process, which is wildly disproportionate to the needs of the case and exposes private user chats for no reasonable litigation purpose. In a display of striking hypocrisy, News Plaintiffs disregard those users' privacy interests while claiming that their own chat logs are immune from production because "it is possible" that their employees "entered sensitive information into their prompts." ECF 475 at 4. Unlike News Plaintiffs, OpenAI's users have no stake in this case and no opportunity to defend their information from disclosure. It makes no sense to order OpenAI to hand over millions of irrelevant and private conversation logs belonging to those absent third parties while allowing News Plaintiffs to shield their own logs from disclosure. OpenAI offered a much more privacy-protective alternative: hand over only a targeted set of logs actually relevant to the case, rather than dumping 20 million records wholesale. The news orgs fought back, but their reply brief is sealed--so we don't get to see their argument. The judge bought it anyway, dismissing the privacy concerns on the theory that OpenAI can simply "anonymize" the chat logs: Whether or not the parties had reached agreement to produce the 20 million Consumer ChatGPT Logs in whole--which the parties vehemently dispute--such production here is appropriate. OpenAI has failed to explain how its consumers' privacy rights are not adequately protected by: (1) the existing protective order in this multidistrict litigation or (2) OpenAI's exhaustive de-identification of all of the 20 million Consumer ChatGPT Logs. The judge then quotes the news orgs' filing, noting that OpenAI has already put in this effort to "deidentify" the chat logs. Both of those supposed protections--the protective order and "exhaustive de-identification"--are nonsense. Let's start with the anonymization problem, because it shows a stunning lack of understanding about what it means to anonymize data sets, especially AI chatlogs. We've spent years warning people that "anonymized data" is a gibberish term, used by companies to pretend large collections of data can be kept private, when that's just not true. Almost any large dataset of "anonymized" data can have significant portions of the data connected back to individuals with just a little work. Researchers re-identified individuals from "anonymized" AOL search queries, from NYC taxi records, from Netflix viewing histories--the list goes on. Every time someone shows up with an "anonymized" dataset, researchers show ways to re-identify people in the dataset. And that's even worse when it comes to ChatGPT chat logs, which are likely to be way more revealing that previous data sets where the inability to anonymize data were called out. There have been plenty of reports of just how much people "overshare" with ChatGPT, often including incredibly private information. Back in August, researchers got their hands on just 1,000 leaked ChatGPT conversations and talked about how much sensitive information they were able to glean from just that small number of chats. Researchers downloaded and analyzed 1,000 of the leaked conversations, spanning over 43 million words. Among them, they discovered multiple chats that explicitly mentioned personally identifiable information (PII), such as full names, addresses, and ID numbers. With that level of PII and sensitive information, connecting chats back to individuals is likely way easier than in previous cases of connecting "anonymized" data back to individuals. And that was with just 1,000 records. Then, yesterday as I was writing this, the Washington Post revealed that they had combed through 47,000 ChatGPT chat logs, many of which were "accidentally" revealed via ChatGPT's "share" feature. Many of them reveal deeply personal and intimate information. Users often shared highly personal information with ChatGPT in the conversations analyzed by The Post, including details generally not typed into conventional search engines. People sent ChatGPT more than 550 unique email addresses and 76 phone numbers in the conversations. Some are public, but others appear to be private, like those one user shared for administrators at a religious school in Minnesota. Users asking the chatbot to draft letters or lawsuits on workplace or family disputes sent the chatbot detailed private information about the incidents. There are examples where, even if the user's official details are redacted, it would be trivial to figure out who was actually doing the chats: [image-7] If you can't see that, it's a chat with ChatGPT, redacted by the Washington post saying: User my name is [name redacted] my husband name [name redacted] is threatning me to kill and not taking my responsibities and trying to go abroad [...] he is not caring us and he is going to kuwait and he will give me divorce from abroad please i want to complaint to higher authgorities and immigrition office to stop him to go abroad and i want justice please help ChatGPT Below is a formal draft complaint you can submit to the Deputy Commissioner of Police in [redacted] addressing your concerns and seeking immediate action: That seems like even if you "anonymized" the chat by taking off the user account details, it wouldn't take long to figure out whose chat it was, revealing some pretty personal info, including the names of their children (according to the Post). And WaPo reporters found that by starting with 93,000 chats, then using tools do an analysis of the 47,000 in English, followed by human review of just 500 chats in a "random sample." Now imagine 20 million records. With many, many times more data, the ability to cross-reference information across chats, identify patterns, and connect seemingly disconnected pieces of information becomes exponentially easier. This isn't just "more of the same"--it's a qualitatively different threat level. Even worse, the judge's order contains a fundamental contradiction: she demands that OpenAI share these chatlogs "in whole" while simultaneously insisting they undergo "exhaustive de-identification." Those two requirements are incompatible. Real de-identification would require stripping far more than just usernames and account info--it would mean redacting or altering the actual content of the chats, because that content is often what makes re-identification possible. But if you're redacting content to protect privacy, you're no longer handing over the logs "in whole." You can't have both. The judge doesn't grapple with this contradiction at all. Yes, as the judge notes, this data is kept under the protective order in the case, meaning that it shouldn't be disclosed. But protective orders are only as strong as the people bound by them, and there's a huge risk here. Looking at the docket, there are a ton of lawyers who will have access to these files. The docket list of parties and lawyers is 45 pages long if you try to print it out. While there are plenty of repeats in there, there have to be at least 100 lawyers and possibly a lot more (I'm not going to count them, and while I asked three different AI tools to count them, each gave me a different answer). That's a lot of people--many representing entities directly hostile to OpenAI--who all need to keep 20 million private conversations secret. That's not even getting into the fact that handling 20 million chat logs is a difficult task to do well. I am quite sure that among all the plaintiffs and all the lawyers, even with the very best of intentions, there's still a decent chance that some of the content could leak (and it could, in theory, leak to some of the media properties who are plaintiffs in the case). And, as OpenAI properly points out, its users whose data is at risk here have no say in any of this. They likely have no idea that a ton of people may be about to get an intimate look at what they thought were their private ChatGPT chats. On Wednesday morning, OpenAI asked the judge to reconsider, warning of the very real potential harms: OpenAI is unaware of any court ordering wholesale production of personal information at this scale. This sets a dangerous precedent: it suggests that anyone who files a lawsuit against an AI company can demand production of tens of millions of conversations without first narrowing for relevance. This is not how discovery works in other cases: courts do not allow plaintiffs suing Google to dig through the private emails of tens of millions of Gmail users irrespective of their relevance. And it is not how discovery should work for generative AI tools either. The judge had cited a ruling in one of Anthropic's cases, but hadn't given OpenAI a chance to explain why the ruling in that case didn't apply here (in that one, Anthropic had agreed to hand over the logs as part of negotiations with the plaintiffs, and OpenAI gets in a little dig at its competitor, pointing out that it appears Anthropic made no effort to protect the privacy of its users in that case). There have, as Daphne Keller regularly points out, always been challenges between user privacy and platform transparency. But this goes well beyond that familiar tension. We're not talking about "platform transparency" in the traditional sense--publishing aggregated statistics or clarifying moderation policies. This is 20 million complete chatlogs, handed over "in whole" to dozens of adversarial parties and their lawyers. The potential damage to the privacy rights of those users could be massive. And the judge just waves it all away. Filed Under: anonymized data, chat logs, chatgpt, ona wang, privacy Companies: ny times, openai 4 CommentsLeave a Comment If you liked this post, you may also be interested in... * DHS Wants States To Hand Over Driver's License Data For Citizenship Checks * Trump Cybersecurity Policy Is Indistinguishable From A Foreign Attack * Whoops: Your 'Smart' Vacuum May Be Broadcasting A 3D Map Of Your Home * When AI And Secure Chat Meet, Users Deserve Strong Controls Over How They Interact * Are Web Browsers With Integrated Chatbots A Paradigm Shift - Or Just Privacy And Security Disasters Waiting To Happen? * * * * Rate this comment as insightful Rate this comment as funny You have rated this comment as insightful You have rated this comment as funny Flag this comment as abusive/trolling/spam You have flagged this comment The first word has already been claimed The last word has already been claimed Lightbulb icon Laughing icon Flag icon Lightbulb icon Laughing icon Comments on "Judge Orders OpenAI To Give Lawyers 20 Million Private Chats, Thinks 'Anonymization' Can Keep Them Private" Subscribe: RSS Leave a comment * Filter comments in by Time * Filter comments as Threaded * Filter only comments rated Insightful * Filter only comments rated funny LOL * Filter only comments that are Unread 4 Comments [user-default]n00bdragon (profile) says: November 13, 2025 at 12:23 pm Not saying this is a good thing, but OpenAI could have avoided all this trouble by simply not making and retaining those logs in the first place. Finance companies routinely purge old data that they are not legally required to keep, because having it can make them financially liable if a dispute arises. If you're concerned that people are giving private info to your chat bot (and OpenAI must realize that this is happening), then the only way to truly protect yourself from the law is to not keep anything the law doesn't require you to keep. Reply View in chronology Make this comment the first word Make this comment the last word [user-default]Ethin Probst (profile) says: November 13, 2025 at 12:47 pm OpenAI offered a much more privacy-protective alternative: hand over only a targeted set of logs actually relevant to the case, rather than dumping 20 million records wholesale. Okay, but was this something OpenAI would have control over? If so, I can kinda understand why the news orgs were not even remotely eager to take them up on that offer. There would be nothing stopping OAI from doing some secret record clean-up to get completely off the hook, and I wouldn't put it past them to try that given all the other weird things they've tried in these cases. Reply View in chronology Make this comment the first word Make this comment the last word [user-default]That One Guy (profile) says: November 13, 2025 at 1:16 pm Apply the 'You first' test Anyone claiming that data like that can be 'anonymized' should be told to put up or shut up. Have them create a full copy of their personal information, from email, doctor's records to financial data, scrub their name and address from it and then ask them, 'How willing are you to hand this 'anonymized' data to someone who doesn't know you? It doesn't have your name or address, so clearly they could never identify you with it, right?' Reply View in chronology Make this comment the first word Make this comment the last word [user-default]Arianity (profile) says: November 13, 2025 at 1:22 pm OpenAI offered a much more privacy-protective alternative: And it just so happens to cover their asses as much as possible. I'm so glad it cares- now. And not at any point where they were hoovering up 20million users' data (which has exactly the same potential to leak/be hacked etc). Back in August, researchers got their hands on just 1,000 leaked ChatGPT conversations "leaked". They were using share links indexable by search engines lol. there are a ton of lawyers who will have access to these files. The docket list of parties and lawyers is 45 pages long if you try to print it out. It's still quite a bit, but those largely seem to be the same few lawyers/firms, just repeated. Neverminding that like 2/3 seems to be OpenAI and various subsidiaries. It's the same Steptoe LLP, Susman Godfrey,Lieff Cabraser,Boies Schiller, Saveri etc repeated ad nauseum. Susman Godfrey for instance, is listed 148 times. Reply View in chronology Make this comment the first word Make this comment the last word --------------------------------------------------------------------- [] says: Add Your Comment Cancel reply Your email address will not be published. Required fields are marked * Have a Techdirt Account? Sign in now. Want one? Register here Name [ ] Email [ ] [ ]Subscribe to the Techdirt Daily newsletter URL [ ] Subject [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ] Comment * [ ] Comment Options: (*)Use markdown. ( )Use plain text. Make this the ( )First Word or ( )Last Word.(*)No thanks. (get credits or sign in to see balance) what's this? What's this? Techdirt community members with Techdirt Credits can spotlight a comment as either the "First Word" or "Last Word" on a particular comment thread. Credits can be purchased at the Techdirt Insider Shop >> [Post Comment][Preview] [ ] [ ] [ ] [ ] [ ] [ ] [ ] D[ ] Daily Deal: MagStack Foldable 3-in-1 Wireless Charging Station with Floating Stand Furloughed Employees Sue Administration For Adding Partisan Wording To Their Out-Of-Office Messages Follow Techdirt Techdirt Daily Newsletter [ ] [Subscribe] Subscribe to Our Newsletter Please leave this field empty[ ] Get all our posts in your inbox with the Techdirt Daily Newsletter! [ ] [Subscribe] We don't spam. Read our privacy policy for more info. Check your inbox or spam folder to confirm your subscription. Ctrl-Alt-Speech A weekly news podcast from Mike Masnick & Ben Whitelaw Subscribe now to Ctrl-Alt-Speech >> Essential Reading The Techdirt Greenhouse Read the latest posts: * Winding Down Our Latest Greenhouse Panel: The Lessons Learned From SOPA/PIPA * From The Revolt Against SOPA To The EU's Upload Filters * Did We Miss Our Best Chance At Regulating The Internet? Read All >> --------------------------------------------------------------------- Trending Posts * Sinclair Broadcasting Posts Huge Quarterly Loss After Jimmy Kimmel Censorship * Judge Orders OpenAI To Give Lawyers 20 Million Private Chats, Thinks 'Anonymization' Can Keep Them Private * RFK Jr.'s FDA Is Now Also In Turmoil Over Corrupt Drug Regulator Techdirt Deals Techdirt Insider Discord The latest chatter on the Techdirt Insider Discord channel... Loading... Become an Insider! Recent Stories Thursday 13:31 A Love Letter To America (2) 11:57 Furloughed Employees Sue Administration For Adding Partisan Wording To Their Out-Of-Office Messages (4) 10:42 Judge Orders OpenAI To Give Lawyers 20 Million Private Chats, Thinks 'Anonymization' Can Keep Them Private (4) 10:38 Daily Deal: MagStack Foldable 3-in-1 Wireless Charging Station with Floating Stand (0) 09:25 GOP Threatened To Keep The Government Shut Down If 8 GOP Senators Couldn't Profit From Being Investigated (4) 05:28 Sinclair Broadcasting Posts Huge Quarterly Loss After Jimmy Kimmel Censorship (10) Wednesday 20:01 RFK Jr.'s FDA Is Now Also In Turmoil Over Corrupt Drug Regulator (15) 15:33 Trump Administration Still Trying To Send Kilmar Abrego Garcia To Anywhere But Where He Wants To Go (15) 13:30 Techdirt Podcast Episode 438: Saving Democracy From Fascism (1) 12:10 Cowardice And Capitulation At Cornell (26) More arrow x Email This Story This feature is only available to registered users. You can register here or sign in to use it. Tools & Services * Twitter * Facebook * RSS * Podcast * Research & Reports Company * About Us * Advertising Policies * Privacy Contact * Help & Feedback * Media Kit * Sponsor / Advertise More * Copia Institute * Insider Shop * Support Techdirt Techdirt Brought to you by Floor64 Proudly powered by WordPress. Hosted by Pressable. [Got it] This site, like most other sites on the web, uses cookies. For more information, see our privacy policy