https://www.wired.com/story/battle-over-books3/ Skip to main content Open Navigation Menu To revist this article, visit My Profile, then View saved stories. Close Alert WIRED The Battle Over Books3 Could Change AI Forever * Backchannel * Business * Culture * Gear * Ideas * Science * Security More To revist this article, visit My Profile, then View saved stories. Close Alert Sign In Search * Backchannel * Business * Culture * Gear * Ideas * Science * Security * Podcasts * Video * Artificial Intelligence * Climate * Games * Newsletters * Magazine * Events * Wired Insider * Jobs * Coupons Kate Knibbs Culture Sep 4, 2023 6:00 AM The Battle Over Books3 Could Change AI Forever Copyright activists are on a mission to wipe a popular generative AI training set from the internet. Success could alter the industry--and who controls it. * * * * Tall stack of books on an offwhite backdrop Photograph: Henrik Sorensen/Getty Images Save Save After OpenAI released GPT-3 in July 2020, independent artificial intelligence researcher Shawn Presser and a few of his fellow machine-learning enthusiasts set a challenge for themselves: Could they recreate it? "We were like, OK, there's actually not that much standing in the way of us doing this ourselves," Presser says. So what if OpenAI had deep pockets and a head start? The Inventor Behind a Rush of AI Copyright Suits Is Trying to Show His Bot Is Sentient Legal Person The Inventor Behind a Rush of AI Copyright Suits Is Trying to Show His Bot Is Sentient Will Bedingfield The Andy Warhol Copyright Case That Could Transform Generative AI State of the Art The Andy Warhol Copyright Case That Could Transform Generative AI Madeline Ashby The Generative AI Battle Has a Fundamental Flaw AI The Generative AI Battle Has a Fundamental Flaw Will Bedingfield That summer, they pored over papers about GPT-3, strategizing in marathon Discord chats about how to best approximate its training data sets. Presser honed in on the books they needed. Suspecting that one of OpenAI's data sets was sourced from an online "shadow library" like Library Genesis, which offers a vast repository of pirated text, he decided to reverse-engineer what he saw as a potentially similar corpus. It was the right moment for Presser to dive into a new project. Unemployed, he struggled with making it to work on time. He'd get dressed, then fall asleep on the couch. Eventually, he'd get a narcolepsy diagnosis. At the time, he just felt frustrated. He wanted to contribute to society. "I was poking around, Googling 'how to download Library Genesis,'" Presser remembers. He found the website of a data archiving group called The Eye; to his amazement, it was hosting links to books from a shadow library called Bibliotik. "I was like, jackpot." He used a script written by the late open-access activist Aaron Swartz to convert the files he scraped, amassing a library of around 196,000 books, including works by popular authors like Stephen King, Margaret Atwood, and Zadie Smith. The project took him a week from start to finish. Since OpenAI had called its book data sets "Books1" and "Books2," Presser decided to keep the tradition alive: He dubbed his pilfered corpus "Books3." Once Presser had assembled his library, he asked The Eye if it could host Books3, in large part because he and his buddies didn't have the money to do it themselves. "We were just nerdy types doing this mostly out of intellectual curiosity." The data-archiving collective agreed. Books3 went online in October 2020. Books3 started as a passion project by a Midwestern guy going through a weird time. "I poured my soul into the work," he says. He saw it as aligned with the open source movement, a way to democratize access to the kind of data sets OpenAI was already using. Some of his collaborators went on to found the nonprofit artificial intelligence collective Eleuther, and Books3 was released as part of Eleuther's larger data set, The Pile. But Presser remains, at core, a bit player on the fringes of the generative AI boom. Despite his obscurity, the data set Presser created is now at the center of a roiling controversy over the future of artificial intelligence. Books3 swiftly became a popular training data set, and not just among academic researchers and Eleuther--big companies, including Meta and Bloomberg, have trained their large language models with it. (Meta declined to comment on this story. Bloomberg did not respond to questions emailed to its lawyer.) Most Popular * What Ever Happened to the Tiny House Movement? Science What Ever Happened to the Tiny House Movement? Eve Andrews * The Pricey Leica Q3 Delivers Amazing Images Gear The Pricey Leica Q3 Delivers Amazing Images Scott Gilbertson * India's Elite Tech Schools Are a Golden Ticket With a Dark Side Business India's Elite Tech Schools Are a Golden Ticket With a Dark Side Akanksha Singh * Apple's Decision to Kill Its CSAM Photo-Scanning Tool Sparks Fresh Controversy Security Apple's Decision to Kill Its CSAM Photo-Scanning Tool Sparks Fresh Controversy Lily Hay Newman * While Presser sees Books3 as a contribution to science, others view his data set in a far less flattering light, and see him as sincere but deeply misguided. For critics, Books3 isn't a boon to society--instead, it's emblematic of everything wrong with generative AI, a glaring example of how both the rights and preferences of artists are disregarded and disrespected by the AI industry's main players, and something that straight-up shouldn't exist. To that point, one small Danish anti-piracy group is on a mission to wipe Books3 from the internet. The Rights Alliance, which represents the interests of creative workers in Denmark, is taking a multifaceted approach to its quest to obliterate Presser's data set. And it is making a surprising amount of progress, especially considering it has only a handful of people working on the project from its Copenhagen headquarters. After spending a week sifting through the data set ("tedious," says Rights Alliance head of content protection and enforcement Thomas Heldrup, the leader of the crusade), they discovered at least 150 works by authors they represented. Heldrup decided to file Digital Millennium Copyright Act (DMCA) takedown notices against the organizations hosting Books3, including The Eye. These efforts paid off. The Eye did, indeed, take the data set down, as did the research data-sharing site Academic Torrents. This did not permanently remove the data from the internet, of course. But it did make it harder to find. (It also didn't necessarily change any minds within these organizations. Academic Torrents director Joseph Paul Cohen complied with the takedown notice, but he says he doesn't understand the intentions behind it. "The greatest authors have read the books that came before them, so it seems weird that we would expect an AI author to only have read openly licensed works," he says.) Rights Alliance isn't stopping there. It also wants to block sites that host Books3 through the European court system. And in addition to pursuing the data set's distributors, Rights Alliance has companies that have already trained their language models using Books3 in its sights, and it has contacted both Meta and Bloomberg on the issue. While Meta has not responded, Heldrup says that Bloomberg did--and that the company told Rights Alliance it does not plan to train future versions of its BloombergGPT using Books3. Meanwhile, in the US, the Authors Guild has organized an open letter to generative AI companies using copyrighted data sets like Books3. "It is only fair that you compensate us for using our writings, without which AI would be banal and extremely limited," the letter states. It's been signed by more than 10,000 writers, many of whom have works that are contained in Books3. The Guild is also discussing a licensed version of The Pile (which includes Books3) with Eleuther. "The goal is to ensure that going forward the AI companies only use licensed data sets," Authors Guild CEO Mary Rasenberger says via email. Some of these writers are taking the matter into their own hands. In a high-profile lawsuit filed against Meta, comedian Sarah Silverman and other authors allege that the company infringed their copyrights by training its set of large language models on Books3. (Silverman and the writers are also suing OpenAI in a similar case.) Most Popular * What Ever Happened to the Tiny House Movement? Science What Ever Happened to the Tiny House Movement? Eve Andrews * The Pricey Leica Q3 Delivers Amazing Images Gear The Pricey Leica Q3 Delivers Amazing Images Scott Gilbertson * India's Elite Tech Schools Are a Golden Ticket With a Dark Side Business India's Elite Tech Schools Are a Golden Ticket With a Dark Side Akanksha Singh * Apple's Decision to Kill Its CSAM Photo-Scanning Tool Sparks Fresh Controversy Security Apple's Decision to Kill Its CSAM Photo-Scanning Tool Sparks Fresh Controversy Lily Hay Newman * Matthew Butterick, himself a writer and programmer, is one of the lawyers representing Silverman and the other authors in both lawsuits. Along with his co-counsel Joseph Saveri, Butterick has become one of the go-to plaintiffs' lawyers in the nation on cases involving copyright and AI. He sees the widespread practice of training AI on copyrighted data as outrageous, and finds it infuriating that this behavior gets defended with claims that it's democratizing access to information. "Open source doesn't mean you took a bunch of people's shit and gave it away for free," he says. "That's theft." Many legal experts WIRED has spoken with range from uncertain to skeptical that these court cases will succeed. Some believe that companies like Meta may be able to successfully evoke fair use, a doctrine allowing use of copyrighted materials without permission under certain circumstances, to argue that what they've done is aboveboard. (Several also said they believe that if Presser ever had legal action brought against him, he could also claim fair use.) It's unclear if courts would see the pirated origins of data sets like Books3 as relevant to the issue of fair use. To draw a parallel, if Sarah Silverman was suing a human writer for infringing on the copyright for her memoir The Bedwetter--say, someone who wrote a suspiciously similar book called The Bedwetter, Too--how said writer had originally read her work might not factor into the verdict. Whether the defendant had purchased a signed copy or flagrantly shoplifted a dog-eared paperback wouldn't matter during arguments over whether The Bedwetter, Too was a derivative rip-off or a transformative parody. Butterick, for his part, thinks the provenance can factor in: "It speaks to the intentionality of your conduct." Presser knew people would get upset about Books3. "We almost didn't release the data sets at all because of copyright concerns," he says. "We thought that there would possibly be some backlash." Looking back, Presser admits that he could've considered the implications a bit more. ("Authors, I hear you.") But he still insists that releasing Books3 was the right thing to do. In his eyes, it leveled the playing field for smaller companies, researchers, and ordinary people who wanted to create large language models. He believes people who want to delete Books3 are unintentionally advocating for a generative AI landscape dominated solely by Big Tech-affiliated companies like OpenAI. "If you really want to knock Books3 offline, fine. Just go into it with eyes wide open. The world that you're choosing is one where only billion-dollar corporations are able to create these large language models," he says. This is a view shared by many copyright lawyers. "If you're OpenAI or Meta, you have the resources to litigate this until the end of time," says Kieran McCarthy, a lawyer specializing in data-scraping issues. "A small organization is not going to have the resources to do that. So this lack of clarity in the law right now is benefiting the biggest players." Most Popular * What Ever Happened to the Tiny House Movement? Science What Ever Happened to the Tiny House Movement? Eve Andrews * The Pricey Leica Q3 Delivers Amazing Images Gear The Pricey Leica Q3 Delivers Amazing Images Scott Gilbertson * India's Elite Tech Schools Are a Golden Ticket With a Dark Side Business India's Elite Tech Schools Are a Golden Ticket With a Dark Side Akanksha Singh * Apple's Decision to Kill Its CSAM Photo-Scanning Tool Sparks Fresh Controversy Security Apple's Decision to Kill Its CSAM Photo-Scanning Tool Sparks Fresh Controversy Lily Hay Newman * Butterick disagrees. "A lawsuit can stop them," he says. "If we prevail." One thing everyone WIRED spoke with could agree upon? All this increased scrutiny on data sets has made AI's big players shy away from transparency. Meta is the prime example. It openly shared the data sets used to train the first version of its ChatGPT competitor Llama, including Books3. Now, it's tight-lipped about what is used for newer versions. "It behooves these companies to be opaque about their sources," McCarthy says. Knowing they're likely to face lawsuits if they fess up to using copyrighted material in their data training sets is a powerful deterrent. This, in turn, will make it harder for writers to know when their copyright is potentially infringed. Right now, it's up to AI companies whether or not to disclose where their training sets come from. Without that information, it's next to impossible for people to prove that their data was used, let alone ask for it to be removed. While the European Parliament has passed a draft law of AI regulations that would require increased data transparency, those regulations are not yet in effect, and other regions lag far behind. This fight cuts to the heart of the often vicious disagreements about what role AI should have in our world. Copyright law exists to balance the rights granted to creators with the collective right to access information, at least in theory. The battle over Books3 is about what this balance should look like in the age of AI. Presser believes that if OpenAI has access to this kind of data set, the public deserves access to them too. From this perspective, attempts to crack down on Books3 may end up calcifying the industry, preventing smaller companies and researchers from entering without doing much to stop the current big players. Copyright law exists to balance the rights granted to creators with the collective right to access information, at least in theory. The battle over Books3 is about what this balance should look like in the age of AI. Pam Samuelson, a copyright lawyer who co-directs the Berkeley Center for Law and Technology, concurs that a crackdown might benefit big corporations that have already been using the data sets. "You can't do it retroactively," she says. She also thinks regulations may change the landscape of where big players congregate. Countries like Israel and Japan have already adopted lax stances on AI training materials, so tighter rules in the EU or US may promote what she calls "innovation arbitrage," where AI entrepreneurs flock to the nations friendlier to their ideas. The heart of this fight boils down to whether we accept that generative AI training on copyrighted material is an inevitability. This is the stance Stephen King recently took after finding out that his work is in Books3. "Would I forbid the teaching (if that is the word) of my stories to computers? Not even if I could. I might as well be King Canute, forbidding the tide to come in. Or a Luddite trying to stop industrial progress by hammering a steam loom to pieces," he wrote. Idealists who want to wrest back control for creators, like Butterick and Hedrup, aren't yet willing to give up the fight. There's a movement to make generative AI training shift into an opt-in model, where only work that is in the public domain or freely given goes into the data sets. "It doesn't have to just be about scraping data sets off the web without permission," emerging technology researcher Eryk Salvaggio says. If AI companies are pushed to scrap the work they've made on copyrighted materials and begin anew, it would certainly upend the current playing field. (Less certain? Whether it's remotely possible.) In the meantime, there are already stopgap efforts to persuade generative AI groups to respect the wishes of people who wish to keep their work out of data sets. Spawning, a startup devoted to this type of tool, has a search engine called "Have I Been Trained?" that currently allows people to check if their visual work has been used in AI training data sets; it is planning to add support for video, audio, and text next year. It also offers an API that helps companies honor opt-outs. So far, StabilityAI is one of the major players to adopt it, although Spawning CEO Jordan Meyer is optimistic that companies like OpenAI and Meta might one day get on board. And Meyer recently made contact with another potential collaborator: Shawn Presser. After everything, Presser does want to help creative types feel they have some control over where their work ends up. "I think it's totally reasonable for people to be able to say, 'Hey, don't use my stuff,'" he says. "That's like a basic sort of tenet of the internet." Get More From WIRED * Get the best stories from WIRED's iconic archive in your inbox * Our new podcast wants you to Have a Nice Future * The world is going blind. Taiwan offers a warning, and a cure * Adderall shortages are dragging on--can video games help? * The impossible fight to stop Canada's wildfires * How to talk to kids about social media and mental health * Using AI to resurrect the dead will create a burden for the living * See if you take a shine to our picks for the best sunglasses and sun protection [undefined] Kate Knibbs is a senior writer at WIRED, covering culture. She was previously a writer at The Ringer and Gizmodo. Senior Writer * TopicsBooksartificial intelligenceOpenAIChatGPTIntellectual Property machine learning More from WIRED Why the Great AI Backlash Came for a Tiny Startup You've Probably Never Heard Of Why the Great AI Backlash Came for a Tiny Startup You've Probably Never Heard Of A literary analytics project called Prosecraft has shuttered after backlash from the writing community. It's a harbinger of a bigger cultural tide shift. Kate Knibbs AI Can Give You an NPC That Remembers. It Could Also Get Your Favorite Artist Fired AI Can Give You an NPC That Remembers. It Could Also Get Your Favorite Artist Fired As game developers wrestle with the challenges and opportunities of incorporating AI tools across the industry, those same tools continue to increase in complexity. Fernanda Seavon Sexy AI Chatbots Are Creating Thorny Issues for Fandom Sexy AI Chatbots Are Creating Thorny Issues for Fandom Generative AI allows fans to "talk" to their favorite characters, drawing comparisons to everything from role-playing to fan fiction. But do they actually want to outsource all the fun to AI? Elizabeth Minkel The Monstrous Crochet Creations of ChatGPT The Monstrous Crochet Creations of ChatGPT When Alex Woolner asked ChatGPT to generate some crochet patterns for cute animals, she wasn't expecting the results to be quite so ... off. Amit Katwala How an Iowa School District Used ChatGPT to Ban Books How an Iowa School District Used ChatGPT to Ban Books Under a new law, schools in Iowa have to remove titles with specific sexual content from libraries. Asking an AI chatbot proved to be the easiest way. It flagged The Handmaid's Tale and Beloved. Angela Watercutter Hip Hop 2073: A Vision of the Future, 50 Years From Now Hip Hop 2073: A Vision of the Future, 50 Years From Now To honor the genre's 50th anniversary, WIRED contributor C. Brandon Ogbunu and Grammy-winning rapper Lupe Fiasco paint two scenes of how the duality of AI will shape the art form in five decades. C. Brandon Ogbunu The Hip Hop Historians Who Are Racing to Preserve Its Story The Hip Hop Historians Who Are Racing to Preserve Its Story From vinyl to photos, blogs to music streams, chronicling 50 years of culture involves reckoning with what can and cannot be saved--and the tools that can do it. Angela Watercutter The AI Doomsday Bible Is a Book About the Atomic Bomb The AI Doomsday Bible Is a Book About the Atomic Bomb The Making of the Atomic Bomb details how scientists led humanity to a new era of destruction. Artificial intelligence safety researchers say it offers valuable lessons for humanity's current plight. Matt Reynolds WIRED WIRED is where tomorrow is realized. It is the essential source of information and ideas that make sense of a world in constant transformation. The WIRED conversation illuminates how technology is changing every aspect of our lives--from culture to business, science to design. The breakthroughs and innovations that we uncover lead to new ways of thinking, new connections, and new industries. * * * * * * More From WIRED * Subscribe * Newsletters * FAQ * Wired Staff * Press Center * Coupons * Editorial Standards * Archive Contact * Advertise * Contact Us * Customer Care * Jobs * RSS * Accessibility Help * Conde Nast Store * Do Not Sell My Personal Info (c) 2023 Conde Nast. All rights reserved. Use of this site constitutes acceptance of our User Agreement and Privacy Policy and Cookie Statement and Your California Privacy Rights. WIRED may earn a portion of sales from products that are purchased through our site as part of our Affiliate Partnerships with retailers. The material on this site may not be reproduced, distributed, transmitted, cached or otherwise used, except with the prior written permission of Conde Nast. Ad Choices Select international siteUnited States * UK * Italia * Japon