https://www.llnl.gov/article/53626/gordon-bell-finalist-team-pushes-scale-rocket-simulation-el-capitan Skip to main content U.S. flag An official website of the United States government Here's how you know Here's how you know [icon-dot-g] Official websites use .gov A .gov website belongs to an official government organization in the United States. [icon-https] Secure .gov websites use HTTPS A lock ( Locked padlock icon ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites. * Our Purpose * Our Science and Technology * Join Our Team * Partner with Us * News * Community and Education * About LLNL Logo Primary Menu * Our Purpose + Our Mission + Management and Sponsors + Our History o Our History - 1950s o Our History - 1960s o Our History - 1970s o Our History - 1980s o Our History - 1990s o Our History - 2000s o Our History - 2010s o Our History - 2020s * Our Science and Technology + Advanced Materials and Manufacturing + Bioscience & Bioengineering + Earth and Atmospheric Science + Facilities and Centers + High-Energy-Density Science + HPC, Simulation and Data Science + Lasers and Optical Science and Technology + Nuclear, Chemical and Isotopic Science and Technology * Join Our Team + Careers o Career Areas o Students o Postdocs o Events & Resources o LCA Notifications o Affiliate Opportunities o Accessibility + Culture * Partner With Us + Office of Government and External Affairs * News + Media Contacts + Media Library + Publications + Lab Report + Fact Sheets + Social Media * Community and Education + Discovery Center + Community Giving + Science Education + Environment, Safety, and Health + Employee and Retirement Resources o Want Ads o In Memoriam + Tours * About LLNL + Our Leadership + Our Impact + By the Numbers + Our Organizations o Operations and Business + Protocol and Special Events + Visiting + Directions + Contact Search [ ] Search Search [ ] Search Mobile Menu Site Menu * Our Purpose * Our Science and Technology * Join Our Team * Partner With Us * News * Community and Education * About LLNL LLNL Menu * Missions * Research * Doing Business * News * About * Careers * Community Our Science and Technology Advanced Materials and Manufacturing Bioscience and Bioengineering Earth and Atmospheric Science Facilities, Centers and Institutes High Energy Density Science HPC, Simulation and Data Science Lasers & Optical Science and Technology Nuclear, Chemical & Isotopic Science and Technology Learn How We Make Science and Technology Happen We support diverse research activities with talented staff, state-of-the-art facilities and core competencies. From internal collaboration to external partnerships, we work together to advance scientific discovery. Learn More About Our Science and Technology News Highlights Media Contacts Media Library Publications Lab Report Fact Sheets Social Media Exascale Computing Comes to LLNL The El Capitan supercomputer will give complex high-fidelity modeling, artificial intelligence and simulation an exascale boost. Learn how it will support national security and stockpile stewardship. Learn More About Exascale Computing Our Science and Technology Advanced Materials and Manufacturing Bioscience and Bioengineering Earth and Atmospheric Science Facilities, Centers and Institutes High Energy Density Science HPC, Simulation and Data Science Lasers & Optical Science and Technology Nuclear, Chemical & Isotopic Science and Technology Learn How We Make Science and Technology Happen We support diverse research activities with talented staff, state-of-the-art facilities and core competencies. From internal collaboration to external partnerships, we work together to advance scientific discovery. Learn More About Our Science and Technology News Highlights Media Contacts Media Library Publications Lab Report Fact Sheets Social Media Exascale Computing Comes to LLNL The El Capitan supercomputer will give complex high-fidelity modeling, artificial intelligence and simulation an exascale boost. Learn how it will support national security and stockpile stewardship. Learn More About Exascale Computing Back Gordon Bell finalist team pushes scale of rocket simulation on El Capitan Examples of rocket plume simulation (Download Image) Researchers used Lawrence Livermore National Laboratory's exascale supercomputer El Capitan to perform the largest fluid dynamics simulation ever -- surpassing one quadrillion degrees of freedom in a single computational fluid dynamics problem. The team focused the effort on rocket-rocket plume interactions, simulating the turbulent exhaust flow generated by many rocket engines firing simultaneously. (Image credit: Spencer Bryngelson/Georgia Tech) Researchers used Lawrence Livermore National Laboratory's (LLNL) exascale supercomputer El Capitan to perform the largest fluid dynamics simulation ever -- surpassing one quadrillion degrees of freedom in a single computational fluid dynamics (CFD) problem. The team focused the effort on rocket-rocket plume interactions. El Capitan is funded by the National Nuclear Security Administration's (NNSA) Advanced Simulation and Computing (ASC) program. The work -- in part performed prior to the transition of the world's most powerful supercomputer to classified operations earlier this year -- is led by researchers from Georgia Tech and supported by partners at AMD, NVIDIA, HPE, Oak Ridge National Laboratory (ORNL) and New York University's (NYU) Courant Institute. The paper is a finalist for the 2025 ACM Gordon Bell Prize, the highest honor in high-performance computing. This year's winner[ ]-- selected from a small handful of finalists -- will be announced at the International Conference for High Performance Computing, Networking, Storage, and Analysis (SC25) in St. Louis on Nov. 20. To tackle the extreme challenge of simulating the turbulent exhaust flow generated by many rocket engines firing simultaneously, the team's approach combined a newly proposed shock-regularization technique called Information Geometric Regularization (IGR), invented and implemented by professors Spencer Bryngelson of Georgia Tech, Florian Schafer of NYU Courant and Ruijia Cao (now a Cornell Ph.D. student). Using all 11,136 nodes and more than 44,500 AMD Instinct MI300A Accelerated Processing Units (APUs) on El Capitan, the team achieved better than 500 trillion grid points, or 500 quadrillion degrees of freedom. They further extended this to ORNL's Frontier, surpassing one quadrillion degrees of freedom. The simulations were conducted with MFC, a permissively licensed open-source code maintained by Bryngelson's group. With these simulations, they represented the full exhaust dynamics of a complex configuration inspired by SpaceX's Super Heavy booster. The simulation sets a new benchmark for exascale CFD performance and memory efficiency. It also paves the way for computation-driven rocket design, replacing costly and limited physical experiments with predictive modeling at unprecedented resolution, according to the team. Georgia Tech's Bryngelson, the project's lead, said the team used specialized techniques to make efficient use of El Cap's architecture. "In my view, this is an intriguing and marked advance in the fluid dynamics field," Bryngelson said. "The method is faster and simpler, uses less energy on El Capitan, and can simulate much larger problems than prior state-of-the-art[] -- orders of magnitude larger." The team accessed El Capitan via prior collaborations with LLNL researchers and worked with LLNL's El Capitan Center of Excellence and HPE to use the machine on the classified network. LLNL facilitated the effort as part of system-scale stress testing ahead of El Capitan's classified deployment, serving as a public example of the full capabilities of the system before it was turned over for classified use in support of the NNSA's core mission of stockpile stewardship. "We supported this work primarily to evaluate El Capitan's scalability and system readiness," said Livermore Computing's Development Environment Group Leader Scott Futral. "The biggest benefit to the ASC program was uncovering system software and hardware issues that only appear when the full machine is exercised. Addressing those challenges was critical to operational readiness." While the actual compute time was relatively short, the bulk of the effort focused on debugging and resolving issues that emerged at full system scale. Futral added that internal LLNL teams, including those working on tsunami early warning and inertial confinement fusion, have also conducted full-system science demonstrations on El Capitan -- efforts that reflect the Lab's broader commitment to mission-relevant exascale computing. A next-generation challenge solved with next-generation hardware As private-sector spaceflight expands, launch vehicles increasingly rely on arrays of compact, high-thrust engines rather than a few massive boosters. This design provides manufacturing advantages, engine redundancy and easier transport, but also creates new challenges. When dozens of engines fire together, their plumes interact in complex ways that can drive searing hot gases back toward the vehicle's base, threatening mission success, researchers said. The solution depends on understanding how those plumes behave across a wide range of conditions. While wind tunnel experiments can test some of the physics, only large-scale simulation can see the full picture at high resolution and under changing atmospheric conditions, engine failures or trajectory shifts. Until now, such simulations were too costly and memory-intensive to run at meaningful scales, especially in an era of multi-rocket boosters, according to the team. To break that barrier, the researchers replaced traditional "shock capturing" methods -- which struggle with high computational cost and complex flow configurations -- with their IGR approach, which reformulates how shock waves are treated in the simulation, enabling a non-diffusive treatment of the same phenomenon. With IGR, more stable results can be computed more efficiently. With IGR in place, the team focused on scale and speed. Their optimized solver took advantage of El Capitan's unified memory APU design and leveraged mixed-precision storage via AMD's new Flang-based compiler to pack more than 100 trillion grid cells into memory without performance degradation. The result was a full-resolution CFD simulation that ran across El Capitan's full system -- roughly 20 times larger than the previous record for this class of problem. The simulation tracked the exhaust from 33 rocket engines emitting exhaust at Mach 10, capturing the moment-to-moment evolution of plume interactions and heat recirculation effects in fine detail. At the heart of the study was El Capitan's unique hardware architecture, which is equipped with four AMD MI300A APUs per node -- each combining CPU and GPU chips directly access the same physical memory. For CFD problems that require simultaneously high memory loads and performant computation, that design proved essential and comparatively harmless compared to the unified memory strategies required by separate-memory systems. The team conducted scaling tests on multiple systems, including ORNL's Frontier and the Swiss National Supercomputing Centre's Alps. Only El Capitan supports a physically shared memory architecture. The system's unified CPU-GPU memory, based on AMD's MI300A architecture, allowed the entire dataset to reside in a single addressable memory space, eliminating data transfer overhead and enabling larger problem sizes. "We needed El Capitan because no other machine in the world could run a problem of this size at full resolution without compromises," said Bryngelson. "The MI300A architecture gave us unified memory with zero performance penalty, so we could store all our simulation data in a single memory space accessible by both CPU and GPU. That eliminated overhead, cut down the memory footprint and let us scale across the full system. El Capitan didn't just make this work possible; it made it efficient." The researchers also achieved an 80-fold speedup over previous methods, reduced memory footprint by a factor of 25 and cut energy-to-solution by more than five times. By combining algorithmic efficiency with El Capitan's chip design, they showed that simulations of this size can be completed in hours, not weeks. Implications for spaceflight and beyond While the simulation focused on rocket exhaust, the underlying method applies to a wide range of high-speed compressible flow problems -- from aircraft noise prediction to biomedical fluid dynamics, the researchers said. The ability to simulate such flows without introducing artificial viscosity or sacrificing resolution could transform modeling across multiple domains and highlights a key design principle behind El Capitan: pairing breakthrough hardware with real-world scientific impact. "From day one, we designed El Capitan to enable mission-scale simulations that were not previously feasible," said Bronis R. de Supinski, chief technology officer for Livermore Computing. "We're always interested in projects that help validate performance and scientific usability at scale. This demonstration provided insight into El Capitan's behavior under real system stress. While we're supporting multiple efforts internally -- including our own Gordon Bell submission -- this was a valuable opportunity to collaborate and learn from an external team with a proven code." Nov. 19, 2025 Contact Jeremy Thomas portrait Jeremy Thomas [email protected] (925) 422-5539 Tags HPC, Simulation, and Data Science Nuclear deterrence Computing Academic Engagement Government Industry Collaborations ASC Livermore Computing Science Space Stockpile stewardship Supercomputing Featured Articles Featured Article LLNL, UT & UCSD win Gordon Bell Prize with exascale tsunami forecasting Featured Article Not all immune cells are created equal Featured Article Experiments show how water could form on exoplanets, disrupting current theories LLNL.GOV * Our Purpose * Our Science and Technology * Join Our Team * Partner with Us * News * Community and Education * About LLNL ORGANIZATIONS * Computing * Engineering * Global Security * National Ignition Facility and Photon Science * Operations and Business * Physical and Life Sciences * Strategic Deterrence RESOURCES * Contact * Visiting * Emergency Communication * Research Ethics * Disclaimer * Copyright and Reuse * Sitemap STAY CONNECTED STAY CONNECTED LLNL.GOV * Our Purpose * Our Science and Technology * Join Our Team * Partner with Us * News * Community and Education * About LLNL ORGANIZATIONS * Computing * Engineering * Global Security * National Ignition Facility and Photon Science * Operations and Business * Physical and Life Sciences * Strategic Deterrence RESOURCES * Contact * Visiting * Emergency Communication * Research Ethics * Disclaimer * Copyright and Reuse * Sitemap STAY CONNECTED LLNL Lawrence Livermore National Laboratory | 7000 East Avenue * Livermore, CA 94550 | LLNL-WEB-458451 Operated by the Lawrence Livermore National Security, LLC for the Department of Energy's National Nuclear Security Administration Learn about the Department of Energy's Vulnerability Disclosure Program LLNS U.S. DOE NNSA Home Disclaimer Privacy Notice * Our Purpose + Our Mission + Management and Sponsors + Our History o Our History - 1950s o Our History - 1960s o Our History - 1970s o Our History - 1980s o Our History - 1990s o Our History - 2000s o Our History - 2010s o Our History - 2020s * Our Science and Technology + Advanced Materials and Manufacturing + Bioscience & Bioengineering + Earth and Atmospheric Science + Facilities and Centers + High-Energy-Density Science + HPC, Simulation and Data Science + Lasers and Optical Science and Technology + Nuclear, Chemical and Isotopic Science and Technology * Join Our Team + Careers o Career Areas o Students o Postdocs o Events & Resources o LCA Notifications o Affiliate Opportunities o Accessibility + Culture * Partner With Us + Office of Government and External Affairs * News + Media Contacts + Media Library + Publications + Lab Report + Fact Sheets + Social Media * Community and Education + Discovery Center + Community Giving + Science Education + Environment, Safety, and Health + Employee and Retirement Resources o Want Ads o In Memoriam + Tours * About LLNL + Our Leadership + Our Impact + By the Numbers + Our Organizations o Operations and Business + Protocol and Special Events + Visiting + Directions + Contact [matomo]