https://spectrum.ieee.org/parallel-processing-unit [ ] IEEE.orgIEEE Xplore Digital LibraryIEEE StandardsMore Sites Sign InJoin IEEE Startup Says It Can Make a 100x Faster CPU Share FOR THE TECHNOLOGY INSIDER Search: [ ] Explore by topic AerospaceArtificial IntelligenceBiomedicalClimate TechComputing Consumer ElectronicsEnergyHistory of TechnologyRoboticsSemiconductors TelecommunicationsTransportation IEEE Spectrum FOR THE TECHNOLOGY INSIDER Topics AerospaceArtificial IntelligenceBiomedicalClimate TechComputing Consumer ElectronicsEnergyHistory of TechnologyRoboticsSemiconductors TelecommunicationsTransportation Sections FeaturesNewsOpinionCareersDIYEngineering Resources More NewslettersPodcastsSpecial ReportsCollectionsExplainersTop Programming LanguagesRobots Guide /IEEE Job Site / For IEEE Members Current IssueMagazine ArchiveThe InstituteThe Institute Archive For IEEE Members Current IssueMagazine ArchiveThe InstituteThe Institute Archive IEEE Spectrum About UsContact UsReprints & Permissions /Advertising / Follow IEEE Spectrum Support IEEE Spectrum IEEE Spectrum is the flagship publication of the IEEE -- the world's largest professional organization devoted to engineering and applied sciences. Our articles, podcasts, and infographics inform our readers about developments in technology, engineering, and science. Join IEEE Subscribe About IEEEContact & SupportAccessibilityNondiscrimination PolicyTerms IEEE Privacy PolicyCookie PreferencesAd Privacy Options (c) Copyright 2024 IEEE -- All rights reserved. A not-for-profit organization, IEEE is the world's largest technical professional organization dedicated to advancing technology for the benefit of humanity. Enjoy more free content and benefits by creating an account Saving articles to read later requires an IEEE Spectrum account The Institute content is only available for members Downloading full PDF issues is exclusive for IEEE Members Downloading this e-book is exclusive for IEEE Members Access to Spectrum 's Digital Edition is exclusive for IEEE Members Following topics is a feature exclusive for IEEE Members Adding your response to an article requires an IEEE Spectrum account Create an account to access more content and features on IEEE Spectrum , including the ability to save articles to read later, download Spectrum Collections, and participate in conversations with readers and editors. For more exclusive content and features, consider Joining IEEE . Join the world's largest professional organization devoted to engineering and applied sciences and get access to all of Spectrum's articles, archives, PDF downloads, and other benefits. Learn more about IEEE - Join the world's largest professional organization devoted to engineering and applied sciences and get access to this e-book plus all of IEEE Spectrum's articles, archives, PDF downloads, and other benefits. Learn more about IEEE - CREATE AN ACCOUNTSIGN IN JOIN IEEESIGN IN Close Access Thousands of Articles -- Completely Free Create an account and get exclusive content and features: Save articles, download collections, and talk to tech insiders -- all free! For full access and benefits, join IEEE as a paying member. CREATE AN ACCOUNTSIGN IN ComputingNews Startup Says It Can Make a 100x Faster CPU Flow Computing aims to boost central processing units with their 'parallel processing units' Dina Genkina 20 Sep 2024 3 min read Dina Genkina is the computing and hardware editor at IEEE Spectrum An abstract rendering of a gray tiled slab floating in air. Flow Computing In an era of fast-evolving AI accelerators, general purpose CPUs don't get a lot of love. "If you look at the CPU generation by generation, you see incremental improvements," says Timo Valtonen, CEO and co-founder of Finland-based Flow Computing. Valtonen's goal is to put CPUs back in their rightful, 'central' role. In order to do that, he and his team are proposing a new paradigm. Instead of trying to speed up computation by putting 16 identical CPU cores into, say, a laptop, a manufacturer could put 4 standard CPU cores and 64 of Flow Computing's so-called parallel processing unit (PPU) cores into the same footprint, and achieve up to 100 times better performance. Valtonen and his collaborators laid out their case at the Hot Chips conference in August. The PPU provides a speed-up in cases where the computing task is parallelizable, but a traditional CPU isn't well equipped to take advantage of that parallelism, yet offloading to something like a GPU would be too costly. "Typically, we say, 'okay, parallelization is only worthwhile if we have a large workload,' because otherwise the overhead kills lot of our gains," says Jorg Keller, professor and chair of parallelism and VLSI at FernUniversitat in Hagen, Germany, who is not affiliated with Flow Computing. "And this now changes towards smaller workloads, which means that there are more places in the code where you can apply this parallelization." Computing tasks can roughly be broken up into two categories: sequential tasks, where each step depends on the outcome of a previous step, and parallel tasks, which can be done independently. Flow Computing CTO and co-founder Martti Forsell says a single architecture cannot be optimized for both types of tasks. So, the idea is to have separate units that are optimized for each type of task. "When we have a sequential workload as part of the code, then the CPU part will execute it. And when it comes to parallel parts, then the CPU will assign that part to PPU. Then we have the best of both words," Forsell says. According to Forsell, there are four main requirements for a computer architecture that's optimized for parallelism: tolerating memory latency, which means finding ways to not just sit idle while the next piece of data is being loaded from memory; sufficient bandwidth for communication between so-called threads, chains of processor instructions that are running in parallel; efficient synchronization, which means making sure the parallel parts of the code execute in the correct order; and low-level parallelism, or the ability to use the multiple functional units that actually perform mathematical and logical operations simultaneously. For Flow Computing new approach, "we have redesigned, or started designing an architecture from scratch, from the beginning, for parallel computation," Forsell says. Any CPU can be potentially upgraded To hide the latency of memory access, the PPU implements multi-threading: when each thread calls to memory, another thread can start running while the first thread waits for a response. To optimize bandwidth, the PPU is equipped with a flexible communication network, such that any functional unit can talk to any other one as needed, also allowing for low-level parallelism. To deal with synchronization delays, it utilizes a proprietary algorithm called wave synchronization that is claimed to be up to 10,000 times more efficient than traditional synchronization protocols. To demonstrate the power of the PPU, Forsell and his collaborators built a proof-of-concept FPGA implementation of their design. The team says that the FPGA performed identically to their simulator, demonstrating that the PPU is functioning as expected. The team performed several comparison studies between their PPU design and existing CPUS. "Up to 100x [improvement] was reached in our preliminary performance comparisons assuming that there would be a silicon implementation of a Flow PPU running at the same speed as one of the compared commercial processors and using our microarchitecture," Forsell says. Now, the team is working on a compiler for their PPU, as well as looking for partners in the CPU production space. They are hoping that a large CPU manufacturer will be interested in their product, so that they could work on a co-design. Their PPU can be implemented with any instruction set architecture, so any CPU can be potentially upgraded. "Now is really the time for this technology to go to market," says Keller. "Because now we have the necessity of energy efficient computing in mobile devices, and at the same time, we have the need for high computational performance." From Your Site Articles * Multicore CPU: Processor Proliferation > * The Case for Running AI on CPUs Isn't Dead Yet > Related Articles Around the Web * Flow parallel processing unit claims possible 100X CPU increases ... > * Flow Computing is the enabler of next generation SuperCPUs > computer architecturecpumicroarchitectureparallel processing {"imageShortcodeIds":[]} Dina Genkina Dina Genkina is an associate editor at IEEE Spectrum focused on computing and hardware. She holds a PhD in atomic physics and lives in Brooklyn. The Conversation (0) A man pushes a drone with cargo on top of it up a flight of stairs RoboticsNews Finally, A Flying Car(t) 21 Sep 2024 3 min read A small drone descends out of the body of a larger drone which is attached underneath a charging dock. RoboticsNews Video Friday: Zipline Delivers 20 Sep 2024 3 min read light and dark colored blocks stacked on top of each other forming the letters "AI" and connected together with strings against a gradient gray background The InstituteArtificial IntelligenceNews IEEE-USA's New Guide Helps Companies Navigate AI Risks 19 Sep 2024 4 min read Related Stories ComputingSemiconductorsNewsTelecommunications How Amazon's New CPU Fights Cybersecurity Threats Artificial IntelligenceNews The Case for Running AI on CPUs Isn't Dead Yet Artificial IntelligenceNews Machine Learning's New Math