https://www.anandtech.com/show/18915/amd-expands-mi300-family-with-mi300x-gpu-only-192gb-memory [p] [logo2] [ ] [search] Welcome Log out Login Register [icon-twitt] [icon-faceb] [icon-rss] * ABOUT * BENCH * FORUMS * PODCAST [logo_resp] [ ] [search] ABOUT BENCH FORUMS PODCAST LOGIN REGISTER PC ComponentsV * CPUs * GPUs * Motherboards * SSDs * Cases/Cooling/PSUs * Memory * NAS * Storage Smartphones & tabletsV * Smartphones * Tablets * Huawei * HTC * Samsung * Google/Android * Microsoft * Apple * SoCs SystemsV * Notebook Reviews * Desktop Reviews * Mac Reviews * Ultrabooks ENTERPRISE & IT GUIDESV * Best CPUs * Best SSDs * Best Laptops * Best Android Phones * Best Video Cards * Best PSUs * Best Motherboards * Best Gaming Laptops * Best Mechanical Keyboards * Best Consumer HDDs DEALS * Home> GPUs [ ] Menu * PC ComponentsV + CPUs + GPUs + Motherboards + SSDs + Cases/Cooling/PSUs + Memory + NAS + Storage * Smartphones & tabletsV + Smartphones + Tablets + Huawei + HTC + Samsung + Google/Android + Microsoft + Apple + SoCs * SystemsV + Notebook Reviews + Desktop Reviews + Mac Reviews + Ultrabooks * Enterprise & IT * GuidesV + Best CPUs + Best SSDs + Best Laptops + Best Android Phones + Best Video Cards + Best PSUs + Best Motherboards + Best Gaming Laptops + Best Mechanical Keyboards + Best Consumer HDDs * Deals * TRENDING TOPICS * CPUs * Intel * AMD * Storage * GPUs * Motherboards * Smartphones * Mobile * SSDs * Home> * GPUs AMD Expands AI/HPC Product Lineup With Flagship GPU-only Instinct MI300X with 192GB Memory by Ryan Smith on June 13, 2023 2:32 PM EST * Posted in * GPUs * CPUs * AMD * APUs * HPC * Zen 4 * MI300 * CDNA 3 * MI300X 6 Comments | Add A Comment 6 Comments + Add A Comment [AMD-Instin] Alongside their EPYC server CPU updates, as part of today's AMD Data Center event, the company is also offering an update on the status of their nearly-finished AMD Instinct MI300 accelerator family. The company's next-generation HPC-class processors, which use both Zen 4 CPU cores and CDNA 3 GPU cores on a single package, have now become a multi-SKU family of XPUs. Joining the previously announced 128GB MI300 APU, which is now being called the MI300A, AMD is also producing a pure GPU part using the same design. This chip, dubbed the MI300X, uses just CDNA 3 GPU tiles rather than a mix of CPU and GPU tiles in the MI300A, making it a pure, high-performance GPU that gets paired with 192GB of HBM3 memory. Aimed squarely at the large language model market, the MI300X is designed for customers who need all the memory capacity they can get to run the largest of models. First announced back in June of last year, and detailed in greater depth back at CES 2023, the AMD Instinct MI300 is AMD's big play into the AI and HPC market. The unique, server-grade APU packs both Zen 4 CPU cores and CDNA 3 GPU cores on to a single, chiplet-based chip. None of AMD's competitors have (or will have) a combined CPU+GPU product like the MI300 series this year, so it gives AMD an interesting solution with a truly united memory architecture, and plenty of bandwidth between the CPU and GPU tiles. [AMD] MI300 also includes on-chip memory via HBM3, using 8 stacks of the stuff. At the time of the CES reveal, the highest capacity HBM3 stacks were 16GB, yielding a chip design with a maximum local memory pool of 128GB. However, thanks to the recent introduction of 24GB HBM3 stacks, AMD is now going to be able to offer a version of the MI300 with 50% more memory - or 192GB. Which, along with the additional GPU chiplets found on the MI300X, are intended to make it a powerhouse for processing the largest and most complex of LLMs. Under the hood, MI300X is actually a slightly simpler chip than MI300A. AMD has replaced MI300A's trio of CPU chiplets with just two CDNA 3 GPU chiplets, resulting in a 12 chiplet design overall - 8 GPU chiplets and what appears to be another 4 IO memory chiplets. Otherwise, despite excising the CPU cores (and de-APUing the APU), the GPU-only MI300X looks a lot like the MI300A. And clearly, AMD is aiming to take advantage of the synergy in offering both an APU and a flagship CPU in the same package. Raw GPU performance aside (we don't have any hard numbers to speak of right now), a bit part of AMD's story with the MI300X is going to be memory capacity. Just offering a 192GB chip on its own is a big deal, given that memory capacity is the constraining factor for the current generation of large language models (LLMs) for AI. As we've seen with recent developments from NVIDIA and others, AI customers are snapping up GPUs and other accelerators as quickly as they can get them, all the while demanding more memory to run even larger models. So being able to offer a massive, 192GB GPU that uses 8 channels of HBM3 memory is going to be a sizable advantage for AMD in the current market - at least, once MI300X starts shipping. The MI300 family remains on track to ship at some point later this year. According to AMD, the 128GB MI300A APU is already sampling to customers now. Meanwhile the 192GB MI300X GPU will be sampling to customers in Q3 of this year. [AMD] It also goes without saying that, with this announcement, AMD has solidified that they're doing a flexible XPU design at least 3 years before rival Intel. Whereas Intel scrapped their combined CPU+GPU Falcon Shores product for a pure GPU Falcon Shores, AMD is now slated to offer a flexible CPU+GPU/GPU-only product as soon as the end of this year. In this timeframe, it will be going up against products such as NVIDIA's Grace Hopper superchip, which although isn't an APU/ XPU either, comes very close by linking up NVIDIA's Grace CPU with a Hopper GPU via a high bandwidth NVLink. So while we're waiting on further details on MI300X, it should make for a very interesting battle between the two GPU titans. Overall, the pressure on AMD with regards to the MI300 family is significant. Demand for AI accelerators has been through the roof for much of the past year, and MI300 will be AMD's first opportunity to make a significant play for the market. MI300 will not quite be a make-or-break product for the company, but besides getting the technical advantage of being the first to ship a single-chip server APU (and the bragging rights that come with it), it will also give them a fresh product to sell into a market that is buying up all the hardware it can get. In short, MI300 is expected to be AMD's license to print money (ala NVIDIA's H100), or so AMD's eager investors hope. AMD Infinity Architecture Platform Alongside today's 192GB MI300X news, AMD is also briefly announcing what they are calling the AMD Infinity Architecture Platform. This is an 8-way MI300X design, allowing for up to 8 of AMD's top-end GPUs to be interlinked together to work on larger workloads. [AMD] As we've seen with NVIDIA's 8-way HGX boards and Intel's own x8 UBB for Ponte Vecchio, an 8-way processor configuration is currently the sweet spot for high-end servers. This is both for physical design reasons - room to place the chips and room to route cooling through them - as well as the best topologies that are available to link up a large number of chips without putting too many hops between them. If AMD is to go toe-to-toe with NVIDIA and to capture part of the HPC GPU market, then this is one more area where they're going to need to match NVIDIA's hardware offerings AMD is calling the Infinity Architecture Platform an "industry-standard" design. Accoding to AMD, they're using an OCP server platform as their base here; and while this implies that MI300X is using an OAM form factor, we're still waiting to get explicit confirmation of this. Gallery: AMD Instinct MI300 Family * [AMD] * [AMD] * [AMD] * [AMD] * [AMD] * [AMD] PRINT THIS ARTICLE * Post Your Comment [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ] [ ] Please log in or sign up to comment. [ ] [Submit Comment] [ajax-loade] POST A COMMENT 6 Comments View All Comments * lemurbutton - Tuesday, June 13, 2023 - link Congratulations to AMD. Its MI300X GPU will match the M2 Ultra in GPU memory capacity. Reply * mdriftmeyer - Tuesday, June 13, 2023 - link It surpasses it by 96bps on an 8192 bit bus. In short, it stomps all over it. Reply * hecksagon - Tuesday, June 13, 2023 - link MI300X his 5200 gb/s compared to the M2's 800gb/s. It isn't even close. Reply * Gm2502 - Tuesday, June 13, 2023 - link How to show you know nothing about computer architecture without saying you know nothing. Rates as one of the stupidest comments of the year... Reply * name99 - Tuesday, June 13, 2023 - link It's not a completely stupid comment. At the time of the M2 Ultra reveal, a lot of people were mocking it as useless because it did not have the raw compute of high end nV or AMD. The obvious rejoinder to that is that compute is not the whole story, there is real value in less compute coupled to more RAM. MI shows that this wasn't just copium. AMD ALSO believes there's real value associated with adding massive (ie more than current nV) amounts of RAM to a decent, even if it's not best in the world, amounts of compute. You can squabble about the details: whether Apple doesn't have enough GPU compute, whether AMD's bandwidth is over-specced, whether SLC on Apple captures enough data reuse in the particular case of interest (training LLMs) to effectively match AMD's memory performance. But all of those are details; the single biggest point is that this is a validation of Apple's belief that raw compute ala nV is going in the wrong direction; that compute needs to be matched to a *large enough* RAM to match upcoming problems. Reply * ballsystemlord - Tuesday, June 13, 2023 - link I want one. Now I just need to figure out how to get it without selling both of my kidneys and heart. Reply * 1 PIPELINE STORIES + Submit News AMD: EPYC "Genoa-X" CPUs With 1.1GB of L3 Cache Now Available [dell_revea] PCI Express 7.0 Spec Hits Draft 0.3, 512GBps Connectivity on Track For 2025 Release Top 10 Foundries See Revenue Drop Nearly 15% Year-over-Year * Microsoft to Bring Game Pass Games to NVIDIA's GeForce Now * Western Digital Estimates HAMR HDDs to Emerge in 1.5 Years * TSMC Opens Advanced Backend Packaging Fab for AI and HPC Products * Cooler Master Introduces MasterAir MA824 Stealth 250W Dual-Tower CPU Cooler * Palm-Sized Minisforum EM680 Desktop PC Packs AMD's Ryzen 7 6800U and USB4 * SK Hynix Starts Production of 238-Layer 3D NAND: 34% Denser with 2.4 GT/s Interface * Big Leap for Hard Drive Capacities: 32 TB HAMR Drives Due Soon, 40TB on Horizon * Realtek Accuses MediaTek of Conspiring Against It with Patent Hoarder * Intel Expands ProViz Offerings with Arc Pro A60, Arc Pro A60M Graphics Cards TWEETS [twiiter_bi] * gavbon86: RT @RyanSmithAT: List price of a flagship, 96 core CPU with 1152MB of L3 cache: $14,756. I hope you've been saving your pennies * RyanSmithAT: List price of a flagship, 96 core CPU with 1152MB of L3 cache: $14,756. I hope you've been saving your pennies https:/ /t.co/xQ1WflTt5s * gavbon86: RT @RyanSmithAT: While the chiplet-based design of the MI300 always left the door open to further CPU/GPU mix & matching, I wasn't sure if... * RyanSmithAT: While the chiplet-based design of the MI300 always left the door open to further CPU/GPU mix & matching, I wasn't s... https://t.co/wKZxZskoDF * gavbon86: AMD Intros EPYC 97x4 "Bergamo" CPUs: 128 Zen 4c CPU Cores For Servers, Shipping Now https://t.co/cOoxvT3pjj Tip... https://t.co/etRGICPrD0 * RyanSmithAT: Be sure to join us in about 10 minutes for AMD's data center presentation and our live blog coverage of the event.... https://t.co/8lzIQkYfqv * gavbon86: RT @anandtech: AMD's 'Data Center and AI Technology Premiere' event starts in 30 minutes. Come and join AnandTech's live blog coverage of t... * gavbon86: RT @anandtech: Top 10 Foundries See Revenue Drop Nearly 15% Year-over-Year https://t.co/hX8eqplZ9R https://t.co/ H5UVNwNJeG * RyanSmithAT: Need something to browse while Reddit is on strike? Come check out the AnandTech Forums. We have conversations goin... https://t.co/gyJMOnTAjI * RyanSmithAT: @DanMatte I'm generally satisfied with the current state of affairs. What specifically did you have in mind? Follow @ANANDTECH [twitter_do] LINKS * Home * About * Forums * RSS * Pipeline News * Bench * Terms of Use * Contact Us * * TOPICS * CPUs * Motherboards * SSD/HDD * GPUs * Mobile * Enterprise & IT * Smartphones * Memory * Cases/Cooling/PSU(s) * Displays * Mac * Systems * Cloud * Trade Shows * Guides FOLLOW * [icon-faceb]Facebook * [icon-twitt]Twitter * [icon-rss2]RSS The Most Trusted in Tech Since 1997 * About * Advertising * Privacy Policy purch Copyright (c) 2023. All rights reserved. * TOPICS * FOLLOW * ABOUT [gryBg1] * CPUs * Motherboards * SSD/HDD * GPUs * Mobile * Enterprise & IT * Smartphones * Memory * Cases/Cooling/PSU(s) * Displays * Mac * Systems * Cloud * Trade Shows * Guides [gryBg1] [icon-faceb]Facebook [icon-twitt]Twitter [icon-rss2]RSS [gryBg1] About Advertising Privacy Policy Contact Us Terms of Use Show Full Site Copyright (c) 2023. All rights reserved. BENCH * [bench_li_p]CPU * [bench_li_p]SSD * [gpubench20]GPU 2013 * [bench_li_p]GPU 2012 * [bench_li_p]Smartphone 2011 * [bench_li_p]Mobile TOPICS * CPUs * Motherboards * SSD/HDD * GPUs * Mobile * Enterprise & IT * Smartphones * Memory * Cases/Cooling/PSU(s) * Displays * Mac * Systems * Cloud * Trade Shows * Guides FOLLOW [icon-faceb]Facebook [icon-twitt]Twitter [icon-rss2]RSS ABOUT * About * Advertising * Privacy Policy * Contact Us * Show Full Site * Copyright (c) 2023. All rights reserved. Log in Don't have an account? Sign up now Username [ ] Password [ ] Remember Me [ ] [LOG IN][ajax-loade] Lost your password? We've updated our terms. By continuing to use the site and/or by logging into your account, you agree to the Site's updated Terms of Use and Privacy Policy.