https://techcommunity.microsoft.com/t5/azure-high-performance-computing/azure-announces-new-ai-optimized-vm-series-featuring-amd-s/ba-p/3980770 [MicrosoftL] Microsoft Tech Community Home Community Hubs Community Hubs * Community Hubs Home * Products * Special Topics * Video Hub Close Products (51) Special Topics (28) Video Hub (447) Most Active Hubs Microsoft Excel Microsoft Teams Windows Security, Compliance and Identity Microsoft 365 Outlook SharePoint Azure Exchange Windows Server Intune and Configuration Manager Microsoft Viva .NET Sharing best practices for building any app with .NET. Microsoft FastTrack Best practices and the latest news on Microsoft FastTrack Microsoft Viva The employee experience platform to help people thrive at work Most Active Hubs Education Sector ITOps Talk Microsoft Partner Community AI and Machine Learning Core Infrastructure and Security Microsoft Mechanics Healthcare and Life Sciences Internet of Things (IoT) Public Sector Regional Blogs Mixed Reality Azure Partner Community Expand your Azure partner-to-partner network Microsoft Tech Talks Bringing IT Pros together through In-Person & Virtual events MVP Award Program Find out more about the Microsoft MVP Award Program. Video Hub Azure Exchange Microsoft 365 Microsoft 365 Business Microsoft 365 Enterprise Microsoft Edge Microsoft Outlook Microsoft Teams Security SharePoint Windows Browse All Community Hubs Blogs Blogs Events Events * Events Home * Microsoft Ignite * Microsoft Build * Community Events Microsoft Learn Microsoft Learn * Home * Community * Blog * Azure * Dynamics 365 * Microsoft 365 * Security, Compliance & Identity * Power Platform * Github * Teams * .NET Lounge Lounge * 1.2M Members * 9,715 Online * 308K Discussions Search [Search] [ ] [ ] [ ] [ ] [ ] cancel Turn on suggestions Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type. Showing results for Show only | Search instead for Did you mean: Sign In Sign In [Search] [ ] [ ] [ ] [ ] [ ] cancel Turn on suggestions Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type. Showing results for Show only | Search instead for Did you mean: Home * Home * * Azure * * Azure High Performance Computing (HPC) Blog * * Azure announces new AI optimized VM series featuring AMD's flagship MI300X GPU * Back to Blog * Newer Article * Older Article Azure announces new AI optimized VM series featuring AMD's flagship MI300X GPU * Subscribe to RSS Feed * * Mark as New * Mark as Read * * Bookmark * Subscribe * * Printer Friendly Page * Report Inappropriate Content By MarcCharest Marc Charest Published Nov 15 2023 08:15 AM 10.7K Views MarcCharest MarcCharest Microsoft Nov 15 2023 08:15 AM Azure announces new AI optimized VM series featuring AMD's flagship MI300X GPU Nov 15 2023 08:15 AM In our relentless pursuit of pushing the boundaries of artificial intelligence, we understand that cutting-edge infrastructure and expertise is needed to harness the full potential of advanced AI. At Microsoft, we've amassed a decade of experience in supercomputing and have consistently supported the most demanding AI training and generative inferencing workloads. Today, we're excited to announce the latest milestone in our journey. We've created a virtual machine (VM) with an unprecedented 1.5 TB of high bandwidth memory (HBM) that leverages the power of AMD's flagship MI300X GPU. Our Azure VMs powered with the MI300X GPU give customers even more choices for AI optimized VMs. ND MI300X v5, Azure's AI-focused virtual machine series with largest HBM capacity What differentiates a ND MI300X v5 VM from our existing family of ND-series VMs is the inclusion of 8 x AMD Instinct MI300X GPUs interconnected via Infinity Fabric 3.0 in each VM. MI300X offers industry leading HBM capacity and bandwidth, 192 GB of HBM3 memory per GPU capable of speeds up to 5.2 TB/s. With greater capacity and more bandwidth customers can process larger models faster, using fewer GPUs. On top of this, these virtual machines use the same winning platform as our other ND-series VMs, which includes the latest technologies like: * 400 Gb/s NVIDIA Quantum-2 CX7 InfiniBand per GPU with 3.2Tb/s per VM * 4th Gen Intel Xeon Scalable processors * PCIE Gen5 host-to-GPU interconnect with 64GB/s bandwidth per GPU * 16 Channels of DDR5 DIMMs 1.5 TB of HBM Means Processing Bigger AI Models HBM is essential for AI applications due to its high bandwidth, low power consumption, and compact size. It is ideal for AI applications that need to quickly process vast amounts of data. ND MI300X v5 has the most HBM capacity available, 1.5 TB per VM or 192 GB of HBM per GPU, the highest HBM capacity available in the cloud. This lets customers run the largest, most advanced AI models faster, and with fewer GPUs. In the end, you save power, cost, and time to solution. Microsoft and AMD's Long-Term Partnership ND MI300X v5 is the culmination of a long-term partnership between Microsoft and AMD. Years ago, we built clusters of MI50 and MI100 at Microsoft to optimize the training and inferencing of large models with ROCm on AMD GPUs. ONNX Runtime, DeepSpeed, and MSCCL are all examples of Microsoft frameworks that now support AMD GPUs. Then we built the world's top cloud supercomputer (as of June 2023, June 2023 | TOP500) with VMs featuring AMD MI200 GPUs and have been running several internal production workloads on it in Azure. Today marks the next phase in this journey, the ND MI300X v5 VMs. Customers can seamlessly switch to ND MI300X v5 from our other ND-series VMs as AMD's open software platform, ROCm, contains all the libraries, compilers, runtimes, and tools necessary for accelerating compute-intensive applications. Many of the major machine learning frameworks (like PyTorch, TensorFlow) all have ROCm support built in and are supported in an ND MI300X v5 VM. This seamlessness results in lower engineering costs and faster time-to-market for customers' solutions. AI in Azure's DNA Azure is pioneering AI by embracing a comprehensive approach to silicon diversity and developing the most flexible and extensible AI infrastructure. We offer the best capabilities of our industry partners and give customers more options to meet their unique needs. We are proud to extend our AI and supercomputing capabilities with NDv5 MI300X. ND MI300X v5 is available for early access and part of the Azure portfolio soon, allowing you to unlock the potential of AI at Scale in the cloud. 2 Likes Like You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in. * Comment Resize Editor + height - height Co-Authors MarcCharest MarcCharest Version history Last update: Nov 15 2023 08:52 AM Updated by: MarcCharest Labels Share * Share to LinkedIn * Share to Facebook * Share to Twitter * Share to Reddit * Share to Email Browse What's new * Surface Pro 9 * Surface Laptop 5 * Surface Studio 2+ * Surface Laptop Go 2 * Surface Laptop Studio * Surface Duo 2 * Microsoft 365 * Windows 11 apps Microsoft Store * Account profile * Download Center * Microsoft Store support * Returns * Order tracking * Virtual workshops and training * Microsoft Store Promise * Flexible Payments Education * Microsoft in education * Devices for education * Microsoft Teams for Education * Microsoft 365 Education * Education consultation appointment * Educator training and development * Deals for students and parents * Azure for students Business * Microsoft Cloud * Microsoft Security * Dynamics 365 * Microsoft 365 * Microsoft Power Platform * Microsoft Teams * Microsoft Industry * Small Business Developer & IT * Azure * Developer Center * Documentation * Microsoft Learn * Microsoft Tech Community * Azure Marketplace * AppSource * Visual Studio Company * Careers * About Microsoft * Company news * Privacy at Microsoft * Investors * Diversity and inclusion * Accessibility * Sustainability * Sitemap * Contact Microsoft * Privacy * Manage cookies * Terms of use * Trademarks * Safety & eco * About our ads * (c) Microsoft 2023 *