https://spectrum.ieee.org/openai-not-open [ ] IEEE.orgIEEE Xplore Digital LibraryIEEE StandardsMore Sites Sign InJoin IEEE Llama and ChatGPT Are Not Open-Source Share FOR THE TECHNOLOGY INSIDER Search: [ ] Explore by topic AerospaceArtificial IntelligenceBiomedicalClimate TechComputing Consumer ElectronicsEnergyHistory of TechnologyRoboticsSemiconductors TelecommunicationsTransportation IEEE Spectrum FOR THE TECHNOLOGY INSIDER Topics AerospaceArtificial IntelligenceBiomedicalClimate TechComputing Consumer ElectronicsEnergyHistory of TechnologyRoboticsSemiconductors TelecommunicationsTransportation Sections FeaturesNewsOpinionCareersDIYThe Big PictureEngineering Resources More Special ReportsCollectionsExplainersPodcastsVideosNewslettersTop Programming LanguagesRobots Guide For IEEE Members Current IssueMagazine ArchiveThe InstituteTI Archive For IEEE Members Current IssueMagazine ArchiveThe InstituteTI Archive IEEE Spectrum About UsContact UsReprints & PermissionsAdvertising Follow IEEE Spectrum Support IEEE Spectrum IEEE Spectrum is the flagship publication of the IEEE -- the world's largest professional organization devoted to engineering and applied sciences. Our articles, podcasts, and infographics inform our readers about developments in technology, engineering, and science. Join IEEE Subscribe About IEEEContact & SupportAccessibilityNondiscrimination PolicyTerms IEEE Privacy Policy (c) Copyright 2023 IEEE -- All rights reserved. A not-for-profit organization, IEEE is the world's largest technical professional organization dedicated to advancing technology for the benefit of humanity. IEEE websites place cookies on your device to give you the best user experience. By using our websites, you agree to the placement of these cookies. To learn more, read our Privacy Policy. view privacy policy accept & close Enjoy more free content and benefits by creating an account Saving articles to read later requires an IEEE Spectrum account The Institute content is only available for members Downloading full PDF issues is exclusive for IEEE Members Access to Spectrum's Digital Edition is exclusive for IEEE Members Following topics is a feature exclusive for IEEE Members Adding your response to an article requires an IEEE Spectrum account Create an account to access more content and features on IEEE Spectrum, including the ability to save articles to read later, download Spectrum Collections, and participate in conversations with readers and editors. For more exclusive content and features, consider Joining IEEE. Join the world's largest professional organization devoted to engineering and applied sciences and get access to all of Spectrum's articles, archives, PDF downloads, and other benefits. Learn more - CREATE AN ACCOUNTSIGN IN JOIN IEEESIGN IN Close Access Thousands of Articles -- Completely Free Create an account and get exclusive content and features: Save articles, download collections, and talk to tech insiders -- all free! For full access and benefits, join IEEE as a paying member. CREATE AN ACCOUNTSIGN IN Artificial IntelligenceTopicTypeComputingNews Llama and ChatGPT Are Not Open-Source An assessment of openness among ostensibly open-source LLM models finds that few live up to the claim Michael Nolan 9h 4 min read llama in a cage with a computer background iStock/Getty Images/IEEE Spectrum LLMsopen sourceOpenAImachine learningmetaChatGPTartificial intelligence Social media and advertising-technology company Meta recently released an update to its large language model Llama. Llama 2 was released as open source, providing users access to the model's weights, evaluation code, and documentation. Meta states the open-source release was intended to make the model "accessible to individuals, creators, researchers, and businesses so they can experiment, innovate, and scale their ideas responsibly." However, compared to other open-source LLMs and open-source software packages more generally, Llama 2 is considerably closed off. Though Meta has made the trained model available, it is not sharing the model's training data or the code used to train it. While third parties have been able to create applications that extend on the base model, aspiring developers and researchers have a limited ability to pick apart the model as is. In a recent preprint in arXiv, a group of AI researchers at Radboud University, in Nijmegen, Netherlands, argue that Llama 2 is not the only LLM to be questionably labeled as "open source." In the paper, the scientists present a multidimensional assessment of model openness. They use this rubric to score 15 different nominally open-source LLMs on different aspects of their availability, documentation, and methods of access. The researchers have collected these assessments in an online table that they have since expanded to include 21 different open-source models. Smaller, research-focused models were included in the assessment if they were deemed, as stated in the preprint, "open, sufficiently documented, and released under an open source license." "Meta using the term 'open source' for this is positively misleading." --Mark Dingemanse, Radboud University The scientists started the project when looking for AI models to use in their own teaching and research. "If you write a research paper, you want the results to be reproducible for as long as possible," says Andreas Liesenfeld, one of the preprint's authors and an assistant professor at Radboud. "That's something you would specifically value if you do research using these technologies, right? That's something we did not see, for instance, from ChatGPT"--the chat-bot interface built off of OpenAI's Generative Pretrained Transformer (GPT) LLM series. Despite what may be inferred from its name, OpenAI closed access to much of its research code after launching GPT-4 and receiving a substantial investment from Microsoft earlier this year. text on the left with green, yellow and orange boxes on the rightThe Radboud University team's assessment gives very poor marks to ChatGPT's and Llama's open-source status. (The full table had 20 entries at press time, we show only the top and bottom entries here.) In fact, OpenAI's ChatGPT model has scored the worst out of all models currently assessed on the team's openness table. Of the available statuses--open, partial, and closed--ChatGPT is marked "closed" in all assessments other than "model card"--a standard format for describing the model and its limitations--and "preprint"--whether or not there is an in-depth research paper about the model. For these two, ChatGPT only gets a "partial" grade. Llama 2 is ranked second worst overall with an overall openness ranking only marginally better than that of ChatGPT. AI's Reproducibility Problems Liesenfeld's concerns about the reproducibility of ChatGPT-based research have borne some evidentiary fruit. A separate preprint from scientists at Stanford University and the University of California, Berkeley, recently demonstrated that both GPT-4 and GPT-3.5's performance on reasoning tasks has changed between March and June of this year, mostly for the worse. These changes have occurred without any accompanying announcement from OpenAI. Such changes may prevent the reproduction of any research results produced from the use of these models during that time period. While Liesenfeld and their colleagues determined that several smaller, research-focused models were considerably more open than Llama 2 or ChatGPT, they found that all the models they assessed were closed in two key ways. First, very few of the models gave sufficient detail of the important refinement process required of modern LLM function, also known as reinforcement learning with human feedback (RLHF). This key step, which tunes language models to give useful outputs from the statistical patterns trained into them during model pretraining, appears to be the secret sauce behind contemporary LLM performance. The process is labor-intensive, requiring human-in-the-loop assessment of model outputs during training. The second major issue the researchers point to is the ways commercial LLM releases have avoided the peer review process. While publishing model architecture, training methods, and performance through reviewed conferences or journals is a well-established practice in academic research, ChatGPT and Llama 2 were both released with only a company-hosted preprint document, most likely to protect trade secret details around model structure and training. While the light this project shines on the variable openness of LLMs may in fact push the field toward truly open-source model development, Liesenfeld and colleagues remain wary of commercial model use in academic research. Mark Dingemanse, a coauthor of this report, had a particularly strong assessment of the Llama 2 model: "Meta using the term 'open source' for this is positively misleading: There is no source to be seen, the training data is entirely undocumented, and beyond the glossy charts the technical documentation is really rather poor. We do not know why Meta is so intent on getting everyone into this model, but the history of this company's choices does not inspire confidence. Users beware." From Your Site Articles * Intel and Nvidia Square Off in GPT-3 Time Trials > * When AI's Large Language Models Shrink - IEEE Spectrum > Related Articles Around the Web * Large language model - Wikipedia > LLMsopen sourceOpenAImachine learningmetaChatGPTartificial intelligence Michael Nolan Michael Nolan is a writer and reporter covering developments in neuroscience, neurotechnology, biometric systems and data privacy. Before that, he spent nearly a decade wrangling biomedical data for a number of labs in academia and industry. Before that he received a masters degree in electrical engineering from the University of Rochester. The Conversation (0) A circuit board with integrated circuits and a keyboard is connected to a monitor via a signal converter box. DIYTopicHands OnMagazineType The Soviet-Era, Z80-based Galaksija Dared to Be Different 7h 5 min read Glowing purple optical fiber curled on a black worktable with a grid of holes in it. TelecommunicationsTopicTypeHistory of TechnologyNews The Invention That Let Fiber Optics Span the Globe 26 Jul 2023 5 min read Former Royal Navy Officer Advocates for LGBTQ+ engineers CareersTopicMagazineTypeProfile Former Royal Navy Officer Advocates for LGBTQ+ engineers 26 Jul 2023 5 min read Related Stories Consumer ElectronicsTopicArtificial IntelligenceTypeNews Meta's Llama 2 Elbows Into a Still-Open Field Artificial IntelligenceTopicTypeNews Ask Yourself These Questions About the Next Rogue-AI Story ComputingTopicArtificial IntelligenceTypeSemiconductorsNews Cerebras Introduces Its 2-Exaflop AI Supercomputer