https://www.hpcwire.com/2023/10/05/how-amd-may-get-across-the-cuda-moat/ Search the site [ ] Go Tabor Network: * [d] Datanami * [e] EnterpriseAI * [hpcwj_favi] HPCwire Japan * # QCwire * # HPC & AI Wall Street Toggle navigation [hpcwire_lo] Search the site [ ] Go Since 1987 - Covering the Fastest Computers in the World and the People Who Run Them Since 1987 - Covering the Fastest Computers in the World and the People Who Run Them * Home * Topics + Applications + Cloud + Developer Tools + Interconnects + Middleware + Networks + Processors + Quantum + Storage + Systems + Visualization * Sectors + Academia & Research + Entertainment + Financial Services + Government + Life Sciences + Manufacturing + Oil & Gas + Retail * QCwire Home * QCwire Subscribe * Exascale * Specials + LIVEwire Interviews + SC o 2022 o 2021 o 2020 o 2019 o 2018 o 2017 o 2016 o 2015 o 2014 o 2013 o 2012 o 2011 o 2010 o 2009 o 2008 + ISC o 2023 o 2022 o 2021 o 2020 o 2019 o 2018 o 2017 o 2016 o 2015 o 2014 o 2013 o 2012 o 2011 o 2010 o 2009 o 2008 + People to Watch o 2023 o 2022 o 2021 o 2020 o 2019 o 2018 o 2017 o 2016 o 2015 o 2014 o 2013 o 2012 o 2011 o 2010 + Readers' Choice Awards o 2023 o 2022 o 2021 o 2020 o 2019 o 2018 o 2017 o 2016 o 2015 o 2014 o 2013 o 2012 o 2011 o 2010 + HPC + AI Wall Street o 2018 o 2009 o 2008 + Advanced Scale Forum * Resource Library * Podcast * Events + Events + Advanced Scale Forum + HPC + AI Wall Street o 2022 Digital Event Series * Solution Channels + AWS + QCT * Job Bank * About + About HPCwire + Organizations and Affiliations + Customer Testimonials + HPCwire Editorial Submissions + Subscribe + HPCwire Custom Reprints + Contact Us * Subscribe [AMD-MI300A] How AMD May Get Across the CUDA Moat By Doug Eadline October 5, 2023 When discussing GenAI, the term "GPU" almost always enters the conversation and the topic often moves toward performance and access. Interestingly, the word "GPU" is assumed to mean "Nvidia" products. (As an aside, the popular Nvidia hardware used in GenAI are not technically Graphical Processing Units. I prefer SIMD units.) The association of GenAI and GPUs with Nvidia is no accident. Nvidia has always recognized the need for tools and applications to help grow its market. They have created a very low barrier to getting software tools (e.g., CUDA) and optimized libraries (e.g., cuDNN) for Nvidia hardware. Indeed, Nvidia is known as a hardware company, but as Bryan Catanzaro, VP of Applied Deep Learning Research, Nvidia has stated " Many people don't know this, but Nvidia has more software engineers than hardware engineers." As a result, Nvidia has built a powerful software "moat" around their hardware. While CUDA is not open source, it is freely available and under the firm control of Nvidia. While this situation has benefited Nvidia (As it should. They invested time and money into CUDA), it has created difficulties for those companies and users that want to grab some of the HPC and GenAI market with alternate hardware. Building on the Castle Foundation The number of foundational models developed for GenAI continues to grow. Many of these are "open source" because they can be used and shared freely. (For example, the Llama foundational model from Meta) In addition, they require a large number of resources (both people and machines) to create and are limited mainly to the hyperscalers (AWS, Microsoft Azure, Google Cloud, Meta Platforms, and Apple) that have huge amounts of GPUs available, In addition to the hyperscalers, other companies have invested in hardware (i.e. purchased a massive amount of GPUs) to create their own foundational models. From a research perspective, the models are interesting and can be used for a variety of tasks; however, the expected use and need for even more GenAI computing resources is two fold; 1. Fine-tuning -- Adding domain-specific data to foundational models to make it work for your use case. 2. Inference - Once the model is fine-tuned, it will require resources when used (i.e., asked questions). These tasks are not restricted to hyperscalers and will need accelerated computing, that is, GPUs. The obvious solution is to buy more "unavailable" Nvidia GPUs, and AMD is ready and waiting now that the demand has far outstripped the supply. To be fair, Intel and some other companies are also ready and waiting to sell into this market. The point is that GenAI will continue to squeeze GPU availability as fine-tuning and inference become more pervasive, and any GPU (or accelerator) is better than no GPU. Moving away from Nvidia hardware suggests that other vendor GPUs and accelerators must support CUDA to run many of the models and tools. AMD has made this possible with HIP CUDA conversion tool; however, the best results often seem to use the native tools surrounding the Nvidia castle. The PyTorch Drawbridge In the HPC sector, CUDA-enabled applications rule the GPU-accelerated world. Porting codes can often realize a speed-up of 5-6x when using a GPU and CUDA. (Note: Not all codes can achieve this speed up, and some may not be able to use the GPU hardware.) However, in GenAI, the story is quite different. Initially, TensorFlow was the tool of choice for creating AI applications using GPUs. It works both with CPUs and was accelerated with CUDA for GPUs. This situation is changing rapidly. An alternative to TensorFlow is PyTorch, an open-source machine learning library for developing and training neural network-based deep learning models. Facebook's AI research group primarily develops it. In a recent blog post by Ryan O'Connor, a Developer Educator at AssemblyAI notes that the popular site HuggingFace, (that allows users to download and incorporate trained and tuned state of the art models into application pipelines with just a few lines of code), 92% of models available are PyTorch exclusive. In addition, as shown in Figure One, a comparison of Machine Learning papers shows a significant trend toward PyTorch and away from TensorFlow. [PyTorch-TF-trends-300x178]Figure One: Percentage of papers that utilize PyTorch, TensorFlow, or another framework over time, with data aggregated quarterly, from late 2017, Source: assemblyai.com. Of course, underneath PyTorch are calls to CUDA, but that is not required because PyTorch insulates the user from the underlying GPU architecture. There is also a version of PyTorch that uses AMD ROCm, an open-source software stack for AMD GPU programming. Crossing the CUDA moat for AMD GPUs may be as easy as using PyTorch. Instinct for Inference In both HPC and GenAI, the Nvidia 72-core ARM-based Grace-Hopper superchip with a shared memory H100 GPU (and also the 144-core Grace-Grace version) is highly anticipated. All Nvidia released benchmarks thus far indicate much better performance than the traditional server where the GPU is attached and accessed over the PCIe bus. Grace-Hopper represents an optimized hardware for both HPC and GenAI. It also is expected to find wide use in both fine-tuning and inference. Demand is expected to be high. AMD has had shared memory CPU-GPU designs since 2006 (AMD acquired graphics card company ATI in 2006). Beginning as the "Fusion" brand many AMD x86_64 processors are now implemented as a combined CPU/GPU called an Accelerated Processing Unit (APU). The upcoming Instinct MI300A processor (APU) from AMD will offer competition for Grace-Hopper superchip. It will also power the forthcoming El Capitan at Lawrence Livermore National Laboratory. The Integrated MI300A will provide up to 24 Zen4 cores in combination with a CDNA 3 GPU Architecture and up to 192 GB of HBM3 memory, providing uniform access memory for all the CPU and GPU cores. The chip-wide cache-coherent memory reduces data movement between the CPU and GPU, eliminating the PCIe bus bottleneck and improving performance and power efficiency. AMD is readying the Instinct MI300A for the upcoming inference market. As stated by AMD CEO Lisa Su in a recent article on Yahoo! Finance. "We actually think we will be the industry leader for inference solutions because of some of the choices that we've made in our architecture." For AMD and many other hardware vendors, PyTorch has dropped the drawbridge on the CUDA moat around the foundational models. AMD has the Instinct MI3000A battle wagon ready to go. The hardware battles for the GenAI market will be won by performance, portability, and availability. The AI day is young. Topics: AI, Analysis, Hardware, Processors, Software Sectors: Academia & Research, Community, Entertainment, Financial Services, Government, Life Sciences, Manufacturing, Oil & Gas, Retail , semiconductor, Space & Physics, Weather & Climate Tags: CUDA, GPU, Grace Hopper, HuggingFace, Instinct MI300A, PyTorch, ROCm, TensorFlow Leading Solution Providers Adaptive Computing Altair AMD Ansys Arm Aspen Systems ASUS AWS Ayar Labs CIQ CoolIT Corehive Cornelis Networks DDN Dell Eviden HPE InspireSemi Inspur Intel Lenovo MemVerge Microsoft Motivair NEC Nvidia Panasas Parallel Works Penguin Solution PNY QCT Silicon Mechanics Sylabs TotalCAE * Off The Wire * Industry Headlines * October 6, 2023 * EuroHPC Launches New Call for Tender Targeting Federation of Supercomputers and Quantum Computers * ORNL, Tennessee State University to Partner on Research Education Programs * BQP and Quest Global Partner to Solve Complex Engineering Problems with Quantum Applications * Argonne Lab, Purdue University Agree to Create Joint Research Positions * Israel Joins the EuroHPC Joint Undertaking * TSMC Releases September 2023 Revenue Report * North Dakota's Largest Academic Supercomputing Facility Adopts Open OnDemand October 5, 2023 * ORNL: Evaluating the Cloud for Capability Class Leadership Workloads * Altair Enhances Leadership in Optimization Technology by Acquiring OmniQuest * BSC Champions EU's EUCAIM Initiative: Cancer Image Europe Sets New Standard for AI in Oncology * Infleqtion's Quantum RF Solution Excels at Army C5ISR NetModX23 Assessment * ACCESS Empowers Cancer Researchers with Big Data Analysis Skills at BigCare Workshop * NVIDIA, Intel and Google Alums Launch Lemurian Labs and Secure $9M in Funding to Solve AI's Compute Capacity Limitations * RIKEN and Fujitsu Announce New 64 Qubit Superconducting Quantum Computer * ALCF's Michael E. Papka Honored with Argonne Board of Governors' Distinguished Performance Award * DOE Announces $16M for Research on the DIII-D National User Facility and Small-scale Experiments * Europe's Exascale Supercomputer in Its Starting Blocks October 4, 2023 * OpenHPC Announces the Release of OpenHPC v3.0 * UT's Texas Institute for Electronics and Infleqtion Launch Quantum Manufacturing Center of Excellence * Keshav Pingali Receives Ken Kennedy Award for High Performance and Parallel Computing More Off The Wire x Off The Wire Industry Headlines More Off The Wire ISC 2023 Booth Videos Cornelis Networks @ Dell Technologies @ Intel @ ISC23 ISC23 ISC23 Lenovo @ ISC23 Microsoft @ ISC23 ISC23 Playlist Subscribe to HPCwire's Weekly Update! Be the most informed person in the room! Stay ahead of the tech trends with industry updates delivered to you every week! * The Latest * Editor's Picks AMD MI3000A How AMD May Get Across the CUDA Moat October 5, 2023 When discussing GenAI, the term "GPU" almost always enters the conversation and the topic often moves toward performance and access. Interestingly, the word "GPU" is assumed to mean "Nvidia" products. (As an aside, the p Read more... [AWS-packaged-quantum-processor-405x228] AWS Survey Showcases Quantum Algorithms and Applications October 5, 2023 Somewhat quietly, while quantum hardware developers have been steadily improving today's early quantum computers (scale and error correction), developers of quantum algorithms and applications have also been accelerating Read more... [JSC-JUPITER-Pressemitteilung-Final-1-405x228] EU Grabs ARM for First ExaFLOP Supercomputer, x86 Misses Out October 4, 2023 The configuration of Europe's first exascale supercomputer, Jupiter, has been finalized, and it is a win for Nvidia and a disappointment for x86 chip vendors Intel and AMD. The Jupiter supercomputer, which will cost EUR2 Read more... Shutterstock_1846345465 HPC User Forum: Sustainability at TACC Points to Software October 3, 2023 Recently, Dan Stanzione, Executive Director, TACC and Associate Vice President for Research, UT-Austin, gave a presentation on HPC sustainability at the Fall 2023 HPC Users Forum. The complete set of slides is available Read more... Shutterstock 1927423355 Google's Controversial AI Chip Paper Under Scrutiny Again October 3, 2023 A controversial research paper by Google that claimed the superiority of AI techniques in creating chips is under the microscope for the authenticity of its claims. Science publication Nature is investigating Google's c Read more... AWS Solution Channel Shutterstock 2338659951 VorTech Derisks Innovative Technology to Aid Global Water Sustainability Challenges Using Cloud-Native Simulations on AWS Overview No more than 1 percent of the world's water is readily available fresh water, according to the US Geological Survey. Read more... Visit the AWS Solution Channel Previous: * Introducing a Community Recipe Library for HPC Infrastructure on AWS * How Maxar Builds Short Duration 'Bursty' HPC Workloads on AWS at Scale * How Amazon's Search M5 Team Optimizes Compute Resources and Cost With Fair-Share Scheduling on AWS Batch QCT Solution Channel [2023-0727-HPC-wire-design2-405x228] QCT and Intel Codeveloped QCT DevCloud Program to Jumpstart HPC and AI Development Organizations and developers face a variety of issues in developing and testing HPC and AI applications. Challenges they face can range from simply having access to a wide variety of hardware, frameworks, and toolkits to time spent on installation, development, testing, and troubleshooting which can lead to increases in cost. Read more... Visit the QCT Solution Channel Previous: * QCT DevCloud Provides Comprehensive Environment for HPC & AI Use Cases * A Purpose-Built HPC/AI Solution for Higher Education And Research * QCT DevCloud Lets Developers Build HPC/AI Applications with Ease [Boeing787_shutterstock_205760866-405x228] Rust Busting: IBM and Boeing Battle Corrosion with Simulations on Quantum Computer October 3, 2023 The steady research into developing real-world applications for quantum computing is piling up interesting use cases. Today, IBM reported on work with Boeing to simulate corrosion processes to improve composites used in Read more... AMD MI3000A How AMD May Get Across the CUDA Moat October 5, 2023 When discussing GenAI, the term "GPU" almost always enters the conversation and the topic often moves toward performance and access. Interestingly, the word "GP Read more... [AWS-packaged-quantum-processor-405x228] AWS Survey Showcases Quantum Algorithms and Applications October 5, 2023 Somewhat quietly, while quantum hardware developers have been steadily improving today's early quantum computers (scale and error correction), developers of qua Read more... [JSC-JUPITER-Pressemitteilung-Final-1-405x228] EU Grabs ARM for First ExaFLOP Supercomputer, x86 Misses Out October 4, 2023 The configuration of Europe's first exascale supercomputer, Jupiter, has been finalized, and it is a win for Nvidia and a disappointment for x86 chip vendors In Read more... Shutterstock 1927423355 Google's Controversial AI Chip Paper Under Scrutiny Again October 3, 2023 A controversial research paper by Google that claimed the superiority of AI techniques in creating chips is under the microscope for the authenticity of its cla Read more... [Boeing787_shutterstock_205760866-405x228] Rust Busting: IBM and Boeing Battle Corrosion with Simulations on Quantum Computer October 3, 2023 The steady research into developing real-world applications for quantum computing is piling up interesting use cases. Today, IBM reported on work with Boeing to Read more... [semiconductor-wafer-chip_shutterstock_1931933390_900x-40] Nvidia Delivering New Options for MLPerf and HPC Performance September 28, 2023 As HPCwire reported recently, the latest MLperf benchmarks are out. Not unsurprisingly, Nvidia was the leader across many categories. The HGX H100 GPU systems, which contain eight H100 GPUs, delivered the highest throughput on every MLPerf inference test in this round. Read more... [IonQ-Barium-Quantum-Computer-405x228] IonQ Announces 2 New Quantum Systems; Suggests Quantum Advantage is Nearing September 27, 2023 It's been a busy week for IonQ, the quantum computing start-up focused on developing trapped-ion-based systems. At the Quantum World Congress today, the compa Read more... [open_black_box_shutterstock_Connect-world-405x228] Rethinking 'Open' for AI September 27, 2023 What does "open" mean in the context of AI? Must we accept hidden layers? Do copyrights and patents still hold sway? And do consumers have the right to opt Read more... [HPCwire-lo] CORNELL I-WAY DEMONSTRATION PITS PARASITE AGAINST VICTIM October 6, 1995 Ithaca, NY --Visitors to this year's Supercomputing '95 (SC'95) conference will witness a life-and-death struggle between parasite and victim, using virtual Read more... [HPCwire-lo] SGI POWERS VIRTUAL OPERATING ROOM USED IN SURGEON TRAINING October 6, 1995 Surgery simulations to date have largely been created through the development of dedicated applications requiring considerable programming and computer graphi Read more... [HPCwire-lo] U.S. Will Relax Export Restrictions on Supercomputers October 6, 1995 New York, NY -- U.S. President Bill Clinton has announced that he will definitely relax restrictions on exports of high-performance computers, giving a boost Read more... [HPCwire-lo] Dutch HPC Center Will Have 20 GFlop, 76-Node SP2 Online by 1996 October 6, 1995 Amsterdam, the Netherlands -- SARA, (Stichting Academisch Rekencentrum Amsterdam), Academic Computing Services of Amsterdam recently announced that it has pur Read more... [HPCwire-lo] Cray Delivers J916 Compact Supercomputer to Solvay Chemical October 6, 1995 Eagan, Minn. -- Cray Research Inc. has delivered a Cray J916 low-cost compact supercomputer and Cray's UniChem client/server computational chemistry software Read more... [HPCwire-lo] NEC Laboratory Reviews First Year of Cooperative Projects October 6, 1995 Sankt Augustin, Germany -- NEC C&C (Computers and Communication) Research Laboratory at the GMD Technopark has wrapped up its first year of operation. Read more... [HPCwire-lo] Sun and Sybase Say SQL Server 11 Benchmarks at 4544.60 tpmC October 6, 1995 Mountain View, Calif. -- Sun Microsystems, Inc. and Sybase, Inc. recently announced the first benchmark results for SQL Server 11. The result represents a n Read more... [HPCwire-lo] New Study Says Parallel Processing Market Will Reach $14B in 1999 October 6, 1995 Mountain View, Calif. -- A study by the Palo Alto Management Group (PAMG) indicates the market for parallel processing systems will increase at more than 4 Read more... Leading Solution Providers Adaptive Computing Altair AMD Ansys Arm Aspen Systems ASUS AWS Ayar Labs CIQ CoolIT Corehive Cornelis Networks DDN Dell Eviden HPE InspireSemi Inspur Intel Lenovo MemVerge Microsoft Motivair NEC Nvidia Panasas Parallel Works Penguin Solution PNY QCT Silicon Mechanics Sylabs TotalCAE Contributors Tiffany Trader Tiffany Trader Editorial Director Douglas Eadline Douglas Eadline Managing Editor John Russell John Russell Senior Editor Jaime Hampton Jaime Hampton Contributing Editor Mariana Iriarte Mariana Iriarte Contributing Editor Alex Woodie Alex Woodie Contributing Editor Addison Snell Addison Snell Contributing Editor Drew Jolly Drew Jolly Editorial Assistant Click Here for More Headlines More Editor's Picks [HPCwire-lo] CORNELL I-WAY DEMONSTRATION PITS PARASITE AGAINST VICTIM October 6, 1995 Ithaca, NY --Visitors to this year's Supercomputing '95 (SC'95) conference will witness a life-and-death struggle between parasite and victim, using virtual Read more... [HPCwire-lo] SGI POWERS VIRTUAL OPERATING ROOM USED IN SURGEON TRAINING October 6, 1995 Surgery simulations to date have largely been created through the development of dedicated applications requiring considerable programming and computer graphi Read more... [HPCwire-lo] U.S. Will Relax Export Restrictions on Supercomputers October 6, 1995 New York, NY -- U.S. President Bill Clinton has announced that he will definitely relax restrictions on exports of high-performance computers, giving a boost Read more... [HPCwire-lo] Dutch HPC Center Will Have 20 GFlop, 76-Node SP2 Online by 1996 October 6, 1995 Amsterdam, the Netherlands -- SARA, (Stichting Academisch Rekencentrum Amsterdam), Academic Computing Services of Amsterdam recently announced that it has pur Read more... [HPCwire-lo] Cray Delivers J916 Compact Supercomputer to Solvay Chemical October 6, 1995 Eagan, Minn. -- Cray Research Inc. has delivered a Cray J916 low-cost compact supercomputer and Cray's UniChem client/server computational chemistry software Read more... [HPCwire-lo] NEC Laboratory Reviews First Year of Cooperative Projects October 6, 1995 Sankt Augustin, Germany -- NEC C&C (Computers and Communication) Research Laboratory at the GMD Technopark has wrapped up its first year of operation. Read more... [HPCwire-lo] Sun and Sybase Say SQL Server 11 Benchmarks at 4544.60 tpmC October 6, 1995 Mountain View, Calif. -- Sun Microsystems, Inc. and Sybase, Inc. recently announced the first benchmark results for SQL Server 11. The result represents a n Read more... [HPCwire-lo] New Study Says Parallel Processing Market Will Reach $14B in 1999 October 6, 1995 Mountain View, Calif. -- A study by the Palo Alto Management Group (PAMG) indicates the market for parallel processing systems will increase at more than 4 Read more... ISC 2023 Booth Videos Cornelis Networks @ Dell Technologies @ Intel @ ISC23 ISC23 ISC23 Lenovo @ ISC23 Microsoft @ ISC23 ISC23 Playlist * arrow * Click Here for More Headlines * arrow [hpcwire_lo] Technologies: * Applications * Cloud * Developer Tools * Interconnects * Middleware * Networks * Processors * Storage * Systems * Visualization Sectors: * Academia & Research * Business * Entertainment * Financial Services * Government * Life Sciences * Manufacturing * Oil & Gas * Retail * Exascale * Multimedia * Events * Organizations and Affiliations * Editorial Submissions * Subscribe * About HPCwire * Contact Us * Sitemap * Reprints [tabor_logo] The Information Nexus of Advanced Computing and Data systems for a High Performance World * TCI Home * Our Publications * Solutions * Live Events * Press * Privacy Policy * Cookie Policy * About Tabor Communications * Update Subscription Preferences * California Consumers (c) 2023 HPCwire. All Rights Reserved. A Tabor Communications Publication HPCwire is a registered trademark of Tabor Communications, Inc. Use of this site is governed by our Terms of Use and Privacy Policy. Reproduction in whole or in part in any form or medium without express written permission of Tabor Communications, Inc. is prohibited. HPCwire This website uses cookies to improve your experience. We'll assume you're ok with this, but you can opt-out if you wish.Accept Reject Read More Privacy & Cookies Policy Close Privacy Overview This website uses cookies to improve your experience while you navigate through the website. Out of these, the cookies that are categorized as necessary are stored on your browser as they are essential for the working of basic functionalities of the website. We also use third-party cookies that help us analyze and understand how you use this website. These cookies will be stored in your browser only with your consent. You also have the option to opt-out of these cookies. But opting out of some of these cookies may affect your browsing experience. Necessary [*] Necessary Always Enabled Necessary cookies are absolutely essential for the website to function properly. This category only includes cookies that ensures basic functionalities and security features of the website. These cookies do not store any personal information. Non-necessary [*] Non-necessary Any cookies that may not be particularly necessary for the website to function and is used specifically to collect user personal data via analytics, ads, other embedded contents are termed as non-necessary cookies. It is mandatory to procure user consent prior to running these cookies on your website. SAVE & ACCEPT Share Blogger Delicious Digg Email Facebook Facebook messenger Google Hacker News Line LinkedIn Mix Odnoklassniki PDF Pinterest Pocket Print Reddit Renren Short link SMS Skype Telegram Tumblr Twitter VKontakte wechat Weibo WhatsApp Xing Yahoo! Mail Copy short link [ ]Copy link