https://www.tomshardware.com/tech-industry/artificial-intelligence/jim-keller-criticizes-nvidias-cuda-and-x86-cudas-a-swamp-not-a-moat-x86-was-a-swamp-too Skip to main content (*) ( ) Open menu Close menu Tom's Hardware [ ] Search Search Tom's Hardware [ ] RSS US Edition flag of US flag of UK UK flag of US US flag of Australia Australia flag of Canada Canada * * Reviews * Best Picks * Raspberry Pi * CPUs * GPUs * Coupons * Newsletter * More + News + PC Components + SSDs + Motherboards + PC Building + Monitors + Laptops + Desktops + Cooling + Cases + RAM + Power Supplies + 3D Printers + Peripherals + Overclocking + About Us Forums Trending * IFS Direct 2024 * AMD Ryzen 7 8700G * Nvidia RTX 4080 Super * Intel 14th-Gen Raptor Lake * PCIe 5.0 SSDs When you purchase through links on our site, we may earn an affiliate commission. Here's how it works. 1. Tech Industry 2. Artificial Intelligence Jim Keller criticizes Nvidia's CUDA, x86 -- 'Cuda's a swamp, not a moat. x86 was a swamp too' News By Anton Shilov published 19 February 2024 Jim Keller is not exactly a fan of Nvidia's CUDA. * * * * * * * Comments (41) Jim Keller and Raja Koduri (Image credit: Tom's Hardware) Jim Keller, a legendary processor architect who has worked on x86, Arm, MISC, and RISC-V processors, this weekend criticized Nvidia's CUDA architecture and software stack and likened it to x86, which he called a swamp. He pointed out that even Nvidia itself has multiple special-purpose software packages that rely on open-source frameworks for performance reasons. "CUDA is a swamp, not a moat," Keller wrote in an X post. "x86 was a swamp too. [...] CUDA is not beautiful. It was built by piling on one thing at a time." Cuda's a swamp, not a moat. x86 was a swamp tooFebruary 17, 2024 See more Indeed, just like x86, CUDA has gradually added functionality while maintaining backward compatibility in software and hardware. This makes Nvidia's platform complete and backward compatible, but it affects performance and makes program development harder. Meanwhile, many open-source software development frameworks can be used more efficiently than CUDA. "Basically nobody writes CUDA," wrote Keller in a follow-up post. "If you do write CUDA, it is probably not fast. [...] There is a good reason there is Triton, Tensor RT, Neon, and Mojo." Even Nvidia itself has tools that do not exclusively rely on CUDA. For example, Triton Inference Server is an open-source tool by Nvidia that simplifies deploying AI models at scale, supporting frameworks like TensorFlow, PyTorch, and ONNX. Triton also provides features like model versioning, multi-model serving, and concurrent model execution to optimize the utilization of GPU and CPU resources. Nvidia's TensorRT is a high-performance deep learning inference optimizer and runtime library that accelerates deep learning inference on Nvidia GPUs. TensorRT takes trained models from various frameworks, such as TensorFlow and PyTorch, and optimizes them for deployment, reducing latency and increasing throughput for real-time applications like image classification, object detection, and natural language processing. But although architectures like Arm, CUDA, and x86 might be considered swamps because of their relatively slow evolution, mandated backward compatibility, and bulkiness, these platforms are also not as fragmented as things like GPGPU, which may not be a bad thing at all. It isn't clear what Jim Keller thinks of AMD's ROCm and Intel's OneAPI, but it is clear that even though he spent many years of his life designing x86 architectures, he isn't enamored with its future prospects. His statements also imply that even though he has worked stints at some of the largest chipmakers in the world, including the likes of Apple, Intel, AMD, Broadcom (and now Tenstorrent), we might not see his name on the Nvidia roster any time soon. Stay on the Cutting Edge Join the experts who read Tom's Hardware for the inside track on enthusiast PC tech news -- and have for over 25 years. We'll send breaking news and in-depth reviews of CPUs, GPUs, AI, maker hardware and more straight to your inbox. [ ][ ]Contact me with news and offers from other Future brands[ ]Receive email from us on behalf of our trusted partners or sponsors[Sign me up] By submitting your information you agree to the Terms & Conditions and Privacy Policy and are aged 16 or over. Anton Shilov Anton Shilov Social Links Navigation Freelance News Writer Anton Shilov is a Freelance News Writer at Tom's Hardware US. Over the past couple of decades, he has covered everything from CPUs and GPUs to supercomputers and from modern process technologies and latest fab tools to high-tech industry trends. More about artificial intelligence AI researchers discuss risks and regulation AI researchers discuss risks and potential regulations -- suggest putting the brakes on the compute hardware as one approach Happy times for OpenAI Elon Musk reminisces about the time Jensen Huang donated a DGX-1 to OpenAI, shares photo gallery Latest Nvidia Nvidia rallies to $2 trillion on AI surge, making founder and CEO Jensen Huang the world's 21st richest person See more latest > See all comments (41) [ ] 41 Comments Comment from the forums * NinoPino There is a typo, it is MIPS not MISC. Reply * ezst036 Maybe Jim should stop flapping his lips and see to it that Tenstorrent produces some motherboards. Without some ATX boards, we can't use his RISC-V designs. With that said, we're stuck in the swamp and he isn't supplying any rope so we can try to stop drowning in it. So what do I care what he says? He isn't producing any solutions to my problem that's for sure. Talk is cheap. This is just another installment in the daily drama. Reply * Argolith ezst036 said: Maybe Jim should stop flapping his lips and see to it that Tenstorrent produces some motherboards. Without some ATX boards, we can't use his RISC-V designs. With that said, we're stuck in the swamp and he isn't supplying any rope so we can try to stop drowning in it. So what do I care what he says? He isn't producing any solutions to my problem that's for sure. Talk is cheap. This is just another installment in the daily drama. Sorry if a single guy doesn't break up corporate monopolies on his own. Reply * The Hardcard Tenstorrent does have silicon out. I don't think it's high volume manufacturing yet. Also by "we" you probably need to mean corporate. I think all of these AI startups are chasing data center dollars, so be surprised if there anything selling for less than $25K for a complete running system. And that's for efficiency plays. Any system that they can show is faster than an H100 system and already has a useable software stack will cost far more. Reply * Findecanor People rooting for RISC-V have been rooting for Tenstorrent's announced 8-wide Ascalon cores. But apparently, those are not actually what Tenstorrent is prioritising. They are mostly intended to be "companion processors" to TPUs. Ascalon has been announced to support only RV64GCV, whereas to reach feature parity with x86 and ARM and for RISC-V software to kick off, you'd need RVA23 compliance. So, IMHO, Tenstorrent is also wading around in the fragmented RISC-V swamp instead of helping build solid software foundations. Reply * Pierce2623 Findecanor said: People rooting for RISC-V have been rooting for Tenstorrent's announced 8-wide Ascalon cores. But apparently, those are not actually what Tenstorrent is prioritising. They are mostly intended to be "companion processors" to TPUs. Ascalon has been announced to support only RV64GCV, whereas to reach feature parity with x86 and ARM and for RISC-V software to kick off, you'd need RVA23 compliance. So, IMHO, Tenstorrent is also wading around in the fragmented RISC-V swamp instead of helping build solid software foundations. Fully agreed. At least CISC computing is mostly unified behind one ISA. RISC V is useless until it has a software stack that actually does things. The tenstorrent CPU that people were excited about doesn't even comply well enough with RISC V to run much of a what is a very thin software stack anyways. It basically turned out to be a driver for their AI solution and nothing else. Reply * hsv-compass Pierce2623 said: Fully agreed. At least CISC computing is mostly unified behind one ISA. RISC V is useless until it has a software stack that actually does things. The tenstorrent CPU that people were excited about doesn't even comply well enough with RISC V to run much of a what is a very thin software stack anyways. It basically turned out to be a driver for their AI solution and nothing else. x86, Arm, MIPS all have unified ISA and mature software stack. Never understand the need for just another poorly supported ISA (YARI - yet another RISC ISA). Is MIPS ISA really royalty free (MIPS open)? Reply * bit_user NinoPino said: There is a typo, it is MIPS not MISC. When did he work on MIPS? He co-architected a DEC Alpha, but that was its own (RISC) ISA, and not MIPS-based. I also thought this was a typo, but I'm not sure what was intended. Anyway, I looked it up and MISC is apparently a thing. I'm still unconvinced that was intended, as it doesn't seem very useful to mention in such a brief summary. https://en.wikipedia.org/wiki/Minimal_instruction_set_computer Honestly, it would be easier just to list the places Jim worked (and delivered!) as a chip architect: DEC AMD (twice) Apple Tesla He also briefly oversaw developments at Intel, but that was higher-level and more forward looking than a chip architect role. we might not see his name on the Nvidia roster any time soon. I think that's a safe bet. He surely doesn't need another salaried position. With Tenstorrent, he's clearly decided to throw his hat into the startup game. Plus, at this point, I'm sure Nvidia has quite a deep roster. They don't really need him like his other recent employers did. Reply * bit_user ezst036 said: Maybe Jim should stop flapping his lips and see to it that Tenstorrent produces some motherboards. Without some ATX boards, we can't use his RISC-V designs. First, they have built boards, such as PCIe cards. However, that's for AI chips where you just need to add DRAM, a VRM, and then you have everything you need to plug it into a PC and start using it. For RISC-V, Tenstorrent has focused on just the IP and building CPU tiles. Jim has given talks (you can find them on youtube), where he's extolled the virtues of chiplets. He's said a small company like his shouldn't have to deal with all of the various enablement IP, like memory controllers, PCIe, storage controllers, etc. So, not only are they not building motherboards for their RISC-V cores, they're not even making complete SoCs. I think it totally makes sense. If he's right, you'll be able to use their cores in CPUs and SoCs from others. However, the lead time on such developments is usually a couple years from the point at which the IP is completed and validated. So, I wouldn't expect to see their RISC-V cores in any devices, just yet. ezst036 said: With that said, we're stuck in the swamp and he isn't supplying any rope so we can try to stop drowning in it. Sure, he is! Tenstorrent makes AI accelerators backed by an open API & toolchain and supported in major AI frameworks. I think you need to read between the lines, a little more. His statement is probably meant to counter the narrative Tenstorrent might be facing from potential customers. Maybe they keep hearing the refrain "...but CUDA!", in their sales calls? Just a guess. Reply * bit_user The Hardcard said: I think all of these AI startups are chasing data center dollars, so be surprised if there anything selling for less than $25K Last year, Tenstorrent inked a deal with a LG, where their AI chiplets will be used in future TVs for AI postprocessing and upscaling. https://tenstorrent.com/research/ tenstorrent-partners-with-lg-to-build-ai-and-risc-v-chiplets-for-smart-tvs-of-the-future / While looking that up, I also found a partnership announcement in the automotive industry: https://tenstorrent.com/research/ bos-and-tenstorrent-partner-to-develop-automotive-semiconductor/ Reply * View All 41 Comments Show more comments Most Popular [missing-im]Framework launches first sub-$500 system: Laptop 13 DIY Edition B-Stock features slight cosmetic blemishes By Mark TysonFebruary 22, 2024 [missing-im]New 'Nvidia App' unifies GeForce Experience and ancient Control Panel -- currently in beta, but for how long? By Aaron KlotzFebruary 22, 2024 [missing-im]Brazilian modders double Nvidia RTX 2080 memory but only see a 10% performance boost By Matthew ConnatserFebruary 22, 2024 [missing-im]Surging AI demand sees Nvidia full-year revenue hit $60.9 billion in 2023 By Anton ShilovFebruary 22, 2024 [missing-im]Intel CEO Pat Gelsinger: I hope to build chips for Lisa Su and AMD By Paul AlcornFebruary 21, 2024 [missing-im]Arm unveils next-gen Neoverse CPU cores and compute subsystems -- hoping to entice more custom silicon customers By Anton ShilovFebruary 21, 2024 [missing-im]Cooler Master sues competitors for infringing on patented pump-inside-radiator cooler design By Aaron KlotzFebruary 21, 2024 [missing-im]Xbox chief Phil Spencer reaffirms commitment to physical media despite brick-and-mortar abandonment, Game Pass success By Christopher HarperFebruary 21, 2024 [missing-im]Intel lands foundry deal for custom Microsoft processor -- 18A process tech to be used for 'very exciting platform shift' By Anton ShilovFebruary 21, 2024 [missing-im]Intel announces new roadmap at IFS Direct Connect 2024: New 14A node, Clearwater Forest taped-in, five nodes in four years remains on track By Paul AlcornFebruary 21, 2024 [missing-im]Microsoft begins forced updates to Windows 11 23H2 -- targets PCs running 21H2 and 22H2 By Brandon HillFebruary 21, 2024 TOPICS CUDA Tom's Hardware is part of Future US Inc, an international media group and leading digital publisher. Visit our corporate site. * Terms and conditions * Contact Future's experts * Privacy policy * Cookies policy * Accessibility Statement * Advertise * About us * Coupons * Careers (c) Future US, Inc. Full 7th Floor, 130 West 42nd Street, New York, NY 10036.