https://www.theregister.com/2023/11/05/biden_ai_reporting_thresholds/ # # Sign in / up The Register(r) -- Biting the hand that feeds IT # # # Topics Security Security All SecurityCyber-crimePatchesResearchCSO (X) Off-Prem Off-Prem All Off-PremEdge + IoTChannelPaaS + IaaSSaaS (X) On-Prem On-Prem All On-PremSystemsStorageNetworksHPCPersonal TechCxOPublic Sector (X) Software Software All SoftwareAI + MLApplicationsDatabasesDevOpsOSesVirtualization (X) Offbeat Offbeat All OffbeatDebatesColumnistsScienceGeek's GuideBOFHLegalBootnotesSite NewsAbout Us (X) Special Features Special Features All Special Features Cloud Infrastructure Week Cybersecurity Month Blackhat and DEF CON Sysadmin Month The Reg in Space Emerging Clean Energy Tech Week Spotlight on RSA Energy Efficient Datacenters Vendor Voice Vendor Voice Vendor Voice All Vendor VoiceAmazon Web Services (AWS) Business TransformationDDN Google Cloud Data TransformationGoogle Cloud InfrastructureGoogle WorkspaceHewlett Packard Enterprise: AI & ML solutionsHewlett Packard Enterprise: Edge-to-Cloud PlatformIntel vProVMware (X) Resources Resources Whitepapers Webinars & Events Newsletters [hpc] HPC 2 comment bubble on white Developing AI models or giant GPU clusters? Uncle Sam would like a word 2 comment bubble on white But the astronomical performance thresholds mean few ML operators will be required to report at this rate icon Tobias Mann Sun 5 Nov 2023 // 18:30 UTC # Comment The White House wants to know who is deploying AI compute clusters and training large language models -- but for now only the really, really, big ones. In an executive order signed this week, US President Joe Biden laid out his agenda for ensuring the safe and productive development of AI technologies. Among the directives was a requirement for operators of AI compute clusters and models exceeding certain thresholds to tell Uncle Sam what they run and where they run it. A closer look at the details of that requirement suggests only the very largest ML companies and infrastructure providers will be compelled to detail their activities. [hpc] The administration wants to know about the development of potential dual-use foundation models, what security measures are being made to protect them, and what steps they're using to prevent misuse. Dual use meaning the neural networks can be used in peaceful civilian and non-peaceful military applications. [hpc] [hpc] The White House also wants to know which companies possess, plan to own, or are in the process of building large scale AI clusters, plus the scale of the deployed compute power and location of facilities. A look at the figures So far the White House has only set interim thresholds that trigger reporting obligations. One requires reporting of any model trained using more than 10^26 integer or floating point operations total, or more than 10^23 floating point operations for biological sequence data. The second sets a threshold for compute clusters located in a single datacenter and networked at more than 100Gb/s. Facilities exceeding 10^20 FLOPS of AI training capacity in that second case will be subject to reporting rules. [hpc] That 10^20 FLOPS figure translates to 100 exaFLOPS, which is a lot for one datacenter. Meanwhile the 10^26 figure is the cumulative number of operations used to train a model over a period of time and would be equivalent to a total of 100 million quintillion floating point operations. Researchers at University of California, Berkeley estimate OpenAI's GPT-3 required about 3.1 x 10^23 floating-point operations of compute to train the full 175 billion parameter model. That's well below the White House's reporting threshold for a single model even though GPTs are just the sort of AI the administration professes to worry about. [hpc] "The common consensus seems to be that very few entities are going to be subject to it," Gartner analyst Lydia Clougherty Jones told The Register. "When you're making a category, you do you have a sense of how many may fall into a category, and sometimes they're so broad that it's not even a category at all, it's almost everybody. This is the opposite of that." * Snowflake puts LLMs in the hands of SQL and Python coders * UK bets on Intel CPUs and GPUs, Dell boxen, OpenStack for Dawn supercomputer * UK govt finds PS225M for Isambard-AI supercomputer powered by Nvidia * Desperately seeking GPUs? AWS will let you reserve instances in advance - no refunds By our estimate, individual models that meet the administration's reporting threshold would employ a cluster of 10,000 Nvidia H100s running at their lowest precision with sparsity for about a month. However, many popular large language models, such as GPT-3, were trained at higher precision, which changes the math a bit. Using FP32, that same cluster would need to be run for 7.5 months to reach that limit. The reporting requirement for AI datacenters is just as eyebrow raising, working out to 100 exaFLOPs. Note that neither rule addresses whether those limits are for FP8 calculations or FP64. As we've previously discussed 1 exaFLOPS at FP64 isn't the same as an 1 exaFLOPS at FP32 or FP8. Context matters. Going back to the H100, you'd need a facility with about 25,000 of the Nvidia GPUs -- each good for 3,958 teraFLOPS of sparse FP8 performance, to meet the reporting requirement. However, if you've deployed something like AMD's Instinct MI250X, which doesn't support FP8, you'd need 261,097 GPUs before the Biden administration wants you to fill in its reporting paperwork. The Register is aware of H100 deployments at that scale. GPU-centric cloud operator CoreWeave has deployed about 22,000 H100s. AI infrastructure startup Voltage Park plans to deploy 24,000 H100s. However neither outfit puts all its GPUs in a single datacenter, so might not exceed the reporting threshold. More precise reporting requirements are on the way. The US Secretary of Commerce has been directed to work with the Secretaries of State, Defense, and Energy, as well as the Director of National Intelligence to define and regularly update reporting rules for what systems and models will need to be reported to the Government. That group has been given 90 days to deliver their first set of rules. This is the Biden Administration effectively saying: "We want to mandate something today, but we need 90 days to figure out exactly what those technical conditions should be," Gartner's Clougherty Jones said. In any case, we expect the number of organizations that will have to report their model developments and AI infrastructure build outs to Uncle Sam under the interim rules will be very small. (r) Bootnote Speaking of machine learning and regulations, OpenAI, Google DeepMind, Amazon, Microsoft, Anthropic, Mistral, and Meta on Thursday signed a non-binding agreement with the UK, America, Singapore, Australia, Canada, the EU, Japan, and others (not China). In that pact, the businesses promised to test their powerful ML models for national security and other risks before releasing them to the wider world. It was inked during the AI Summit taking place in the UK this week. Get our Tech Resources # Share More about * AI * Datacenter * Federal government of the United States More like these x More about * AI * Datacenter * Federal government of the United States * Joe Biden * Large Language Model Narrower topics * Cybersecurity and Infrastructure Security Agency * Cybersecurity Information Sharing Act * Disaster recovery * Federal Aviation Administration * Google AI * GPS * GPT-3 * Immigration and Nationality Act of 1965 * IRS * Machine Learning * MCubed * NASA * National Highway Traffic Safety Administration * National Institute of Standards and Technology * National Labor Relations Board * NCSAM * Neural Networks * NLP * Open Compute Project * PUE * Software defined data center * Star Wars * Telecommunications Act of 1996 * Tensor Processing Unit * United States Department of Defense * United States Department of Justice * US Securities and Exchange Commission Broader topics * ChatGPT * Government * Self-driving Car * United States of America More about # Share 2 comment bubble on white COMMENTS More about * AI * Datacenter * Federal government of the United States More like these x More about * AI * Datacenter * Federal government of the United States * Joe Biden * Large Language Model Narrower topics * Cybersecurity and Infrastructure Security Agency * Cybersecurity Information Sharing Act * Disaster recovery * Federal Aviation Administration * Google AI * GPS * GPT-3 * Immigration and Nationality Act of 1965 * IRS * Machine Learning * MCubed * NASA * National Highway Traffic Safety Administration * National Institute of Standards and Technology * National Labor Relations Board * NCSAM * Neural Networks * NLP * Open Compute Project * PUE * Software defined data center * Star Wars * Telecommunications Act of 1996 * Tensor Processing Unit * United States Department of Defense * United States Department of Justice * US Securities and Exchange Commission Broader topics * ChatGPT * Government * Self-driving Car * United States of America TIP US OFF Send us news --------------------------------------------------------------------- Other stories you might like Hyperscale datacenter capacity set to triple because of AI demand And it's going to suck... up more power too AI + ML18 Oct 2023 | 1 AI luminaries call for urgent regulation to head off future threats, but Meta's brainbox boss disagrees Suggest developers spend 'at least one-third of their R&D budget' on safety AI + ML24 Oct 2023 | 18 Yeah, that oughta do the trick, Joe... Biden hopes to tackle AI safety with exec order ML players must alert Uncle Sam if they're training a foundation model, and more AI + ML30 Oct 2023 | 12 Power tools: turning sustainable IT into a business catalyst How new monitoring and management tools can help datacenters root-out infrastructure inefficiencies Sponsored Feature [hpc] UK govt finds PS225M for Isambard-AI supercomputer powered by Nvidia 5,448 GraceHopper superchips and 200PFLOPS gets you somewhere in the global public top ten HPC1 Nov 2023 | 25 Dell cosies up to Meta to tame Llama 2 AI beast on-prem Spitting in the cloud's eye On-Prem31 Oct 2023 | 1 Just one in ten UK orgs have significant AI investment plan Gen AI still in the pilot stage, despite government hopes to be world leader AI + ML30 Oct 2023 | 20 Boffins find AI stumbles when quizzed on the tough stuff Must try harder, D+ AI + ML29 Oct 2023 | 48 'AI divide' across the US leaves economists concerned AI use up 266% in five years but it's still used by less than a quarter of workers AI + ML24 Oct 2023 | 9 UK bets on Intel CPUs and GPUs, Dell boxen, OpenStack for Dawn supercomputer We'd make some kind of Sun sets joke here but it's too early in the morning HPC2 Nov 2023 | 8 Microsoft warns it may 'throttle' its generative AI services for 'excessive' users Updated T&Cs reveal intent to restrict access under undefined circumstances AI + ML2 Nov 2023 | 9 UK convinces nations to sign Bletchley Declaration in bid for AI safety Tech leaders and politicos descend on Britland to thrash out regulation and governance AI + ML1 Nov 2023 | 34 The Register icon Biting the hand that feeds IT About Us* * Contact us * Advertise with us * Who we are Our Websites* * The Next Platform * DevClass * Blocks and Files Your Privacy* * Cookies Policy * Privacy Policy * T's & C's * Do not sell my personal information Situation Publishing Copyright. All rights reserved (c) 1998-2023 no-js