https://arstechnica.com/ai/2024/11/safe-ai-champ-anthropic-teams-up-with-defense-giant-palantir-in-new-deal/ Skip to content Ars Technica home Sections Forum Subscribe * AI * Biz & IT * Cars * Culture * Gaming * Health * Policy * Science * Security * Space * Tech * Feature * Reviews * Store * AI * Biz & IT * Cars * Culture * Gaming * Health * Policy * Science * Security * Space * Tech Forum Subscribe Story text Size [Standard] Width * [Standard] Links [Standard] * Subscribers only Learn more Pin to story Theme * Light * Dark * System Search dialog... Sign In Sign in dialog... Sign in Defense confabulations Claude AI to process secret government data through new Palantir deal Critics worry Anthropic is endangering its "ethical" AI stance due to defense associations. Benj Edwards - Nov 8, 2024 5:08 pm | 10 A digital iris eyeball, blue illustration. A digital iris eyeball, blue illustration. Credit: Yuichiro Chino via Getty Images Credit: Yuichiro Chino via Getty Images Text settings Story text Size [Standard] Width * [Standard] Links [Standard] * Subscribers only Learn more Minimize to nav Anthropic has announced a partnership with Palantir and Amazon Web Services to bring its Claude AI models to unspecified US intelligence and defense agencies. Claude, a family of AI language models similar to those that power ChatGPT, will work within Palantir's platform using AWS hosting to process and analyze data. But some critics have called out the deal as contradictory to Anthropic's widely-publicized "AI safety" aims. On X, former Google co-head of AI ethics Timnit Gebru wrote of Anthropic's new deal with Palantir, "Look at how they care so much about 'existential risks to humanity.'" The partnership makes Claude available within Palantir's Impact Level 6 environment (IL6), a defense-accredited system that handles data critical to national security up to the "secret" classification level. This move follows a broader trend of AI companies seeking defense contracts, with Meta offering its Llama models to defense partners and OpenAI pursuing closer ties with the Defense Department. In a press release, the companies outlined three main tasks for Claude in defense and intelligence settings: performing operations on large volumes of complex data at high speeds, identifying patterns and trends within that data, and streamlining document review and preparation. While the partnership announcement suggests broad potential for AI-powered intelligence analysis, it states that human officials will retain their decision-making authority in these operations. As a reference point for the technology's capabilities, Palantir reported that one (unnamed) American insurance company used 78 AI agents powered by their platform and Claude to reduce an underwriting process from two weeks to three hours. The new collaboration builds on Anthropic's earlier integration of Claude into AWS GovCloud, a service built for government cloud computing. Anthropic, which recently began operations in Europe, has been seeking funding at a valuation up to $40 billion. The company has raised $7.6 billion, with Amazon as its primary investor. An ethical minefield Since its founders started Anthropic in 2021, the company has marketed itself as one that takes an ethics- and safety-focused approach to AI development. The company differentiates itself from competitors like OpenAI by adopting what it calls responsible development practices and self-imposed ethical constraints on its models, such as its "Constitutional AI" system. As Futurism points out, this new defense partnership appears to conflict with Anthropic's public "good guy" persona, and pro-AI pundits on social media are noticing. Frequent AI commentator Nabeel S. Qureshi wrote on X, "Imagine telling the safety-concerned, effective altruist founders of Anthropic in 2021 that a mere three years after founding the company, they'd be signing partnerships to deploy their ~AGI model straight to the military frontlines." Anthropic's "Constitutional AI" logo. Anthropic's "Constitutional AI" logo. Credit: Anthropic / Benj Edwards Anthropic's "Constitutional AI" logo. Credit: Anthropic / Benj Edwards Aside from the implications of working with defense and intelligence agencies, the deal connects Anthropic with Palantir, a controversial company which recently won a $480 million contract to develop an AI-powered target identification system called Maven Smart System for the US Army. Project Maven has sparked criticism within the tech sector over military applications of AI technology. It's worth noting that Anthropic's terms of service do outline specific rules and limitations for government use. These terms permit activities like foreign intelligence analysis and identifying covert influence campaigns, while prohibiting uses such as disinformation, weapons development, censorship, and domestic surveillance. Government agencies that maintain regular communication with Anthropic about their use of Claude may receive broader permissions to use the AI models. Even if Claude is never used to target a human or as part of a weapons system, other issues remain. While its Claude models are highly regarded in the AI community, they (like all LLMs) have the tendency to confabulate, potentially generating incorrect information in a way that is difficult to detect. That's a huge potential problem that could impact Claude's effectiveness with secret government data, and that fact, along with the other associations, has Futurism's Victor Tangermann worried. As he puts it, "It's a disconcerting partnership that sets up the AI industry's growing ties with the US military-industrial complex, a worrying trend that should raise all kinds of alarm bells given the tech's many inherent flaws--and even more so when lives could be at stake." Photo of Benj Edwards Benj Edwards Senior AI Reporter Benj Edwards Senior AI Reporter Benj Edwards is Ars Technica's Senior AI Reporter and founder of the site's dedicated AI beat in 2022. He's also a widely-cited tech historian. In his free time, he writes and records music, collects vintage computers, and enjoys nature. He lives in Raleigh, NC. 10 Comments Comments Forum view Loading Loading comments... Prev story Next story Most Read 1. Listing image for first story in Most Read: After decades, FDA finally moves to pull ineffective decongestant off shelves 1. After decades, FDA finally moves to pull ineffective decongestant off shelves 2. 2. Sega is delisting 60 classic games from Steam, so now's the time to grab them 3. 3. Trump's likely FCC chair wrote Project 2025 chapter on how he'd run the agency 4. 4. Review: M4 and M4 Pro Mac minis are probably Apple's best Mac minis ever 5. 5. Rocket Report: Australia says yes to the launch; Russia delivers for Iran Customize Ars Technica has been separating the signal from the noise for over 25 years. With our unique combination of technical savvy and wide-ranging interest in the technological arts and sciences, Ars is the trusted source in a sea of information. After all, you don't need to know everything, only what's important. More from Ars * About Us * Staff Directory * Newsletters * Ars Videos * General FAQ * RSS Feeds Contact * Contact us * Advertise with us * Reprints Do Not Sell My Personal Information (c) 2024 Conde Nast. All rights reserved. Use of and/or registration on any portion of this site constitutes acceptance of our User Agreement and Privacy Policy and Cookie Statement and Ars Technica Addendum and Your California Privacy Rights. Ars Technica may earn compensation on sales from links on this site. Read our affiliate link policy. The material on this site may not be reproduced, distributed, transmitted, cached or otherwise used, except with the prior written permission of Conde Nast. Ad Choices