https://www.theregister.com/2023/09/01/intel_graph_analytics_chip/ # # Sign in / up The Register(r) -- Biting the hand that feeds IT # # # Topics Security Security All SecurityCyber-crimePatchesResearchCSO (X) Off-Prem Off-Prem All Off-PremEdge + IoTChannelPaaS + IaaSSaaS (X) On-Prem On-Prem All On-PremSystemsStorageNetworksHPCPersonal Tech (X) Software Software All SoftwareAI + MLApplicationsDatabasesDevOpsOSesVirtualization (X) Offbeat Offbeat All OffbeatDebatesColumnistsScienceGeek's GuideBOFHLegalBootnotesSite NewsAbout Us (X) Special Features Special Features All Special Features Blackhat and DEF CON Sysadmin Month The Reg in Space Emerging Clean Energy Tech Week Spotlight on RSA Energy Efficient Datacenters Vendor Voice Vendor Voice Vendor Voice All Vendor VoiceAmazon Web Services (AWS) Business Transformation DataikuDDNGoogle Cloud for StartupsHewlett Packard Enterprise: AI & ML solutionsHewlett Packard Enterprise: Edge-to-Cloud PlatformIntel vProVMware (X) Resources Resources Whitepapers Webinars & Events Newsletters [networks] Networks 13 comment bubble on white Intel shows off 8-core, 528-thread processor with 1TB/s of co-packaged optics 13 comment bubble on white 'Embarrassingly parallel' protoype baked for DARPA to play Six Degrees of Kevin Bacon at massive scale icon Tobias Mann Fri 1 Sep 2023 // 00:47 UTC # Hot Chips Intel has used this week's Hot Chips conference in California to show off a 528-thread processor with 1TB/s silicon photonics interconnects that's designed to chew the largest analytics workloads, while sipping power. For those hoping for a hyper-parallelized Xeon, this isn't that. The chip isn't even x86 based. Instead, it was purpose built using a custom RISC architecture for DARPA's Hierarchical Identity Verify Exploit (HIVE) program. That US military initiative seeks to develop a graph analytics processor capable of churning through streaming data 100 times faster than conventional compute architectures, while also consuming less power. [networks] Graph analytics deal with how data points connect with other data points in complex systems. The example Intel principal engineer Jason Howard pointed to in his Hot Chips presentation was social networks, which might run graph analytics workloads to understand connections among members. [networks] [networks] Think of it as a really high-performance Six Degrees of Separation (or Kevin Bacon) and you'll get the idea. While this might seem a strange thing for DARPA to consider, the government agency feels large scale graph analytics has applications in infrastructure monitoring and cybersecurity. Speaking of Hot Chips, don't miss: * Arm offering Neoverse CPU cores as a complete set of designs to drop into datacenter processors: Overview and technical deep-dive . * Google talking up its TPUv5e AI accelerator: Overview and technical analysis. * Plus: Google readying Arm-compatible AmpereOne-powered virtual machine instances in its cloud, Intel eyeing up AMD's specs for its next-gen Xeon processors, and AMD teasing edge-friendly Epyc chips. The chip is part of Intel's first direct mesh-to-mesh photonic fabric, which uses silicon photonic interconnects to stitch multiple chips together. But rather than using copper to stitch two, four or even eight sockets together using copper traces, Intel used co-packaged optics to connect hundreds or even thousands of chips together in a low latency, high bandwidth mesh. That's the idea anyway, the chip is still a prototype. Scaling up graph analytics with silicon photonics The goal is to develop tech that can be scaled up to support even the largest graph analytics workloads. The four chiplets surrounding the central compute die supply the Intel prototype with 1TB/s of optical bandwidth The four chiplets surrounding the central compute die supply the Intel prototype with 1TB/s of optical bandwidth - Click to enlarge While the chip initially appears to be a standard processor with some extra bits and bobs -- it even has a fairly typical BGA interface not unlike those found on Xeon-D chips -- most of the data flowing in and out of the chip is transmitted optically using silicon photonics chiplets developed in collaboration with Ayar Labs. The four little chiplets surrounding the central processing die convert the electrical signals coming in and out of the microprocessor to optical signals carried by 32 single-mode fibers. As we understand it 16 of those fiber strands are for transmitting data and the other 16 are for receiving it. [networks] According to Intel, each of those fibers can carry data at 32GB/s on and off the chip for an aggregate of 1TB/s of bandwidth. However, Howard said that in testing the team had only managed to achieve half that rate. Most of the data moving in and out of Intel's prototype is transmitted optically. Most of the data moving in and out of Intel's prototype processor is transmitted optically, as you can see ... Click to enlarge As Intel envisions it, 16 of these chips would be networked together in an all-to-all configuration on a single Open Compute Project server in its sled form factor. Then multiple sleds, potentially as many as 100,000, could be networked together again in a sled-to-sled configuration. The result would be that any one chip could communicate with another, at very low latencies, regardless of which sled it resides in. That's not to say the x86 goliath didn't run into challenges getting the optics to cooperate. In addition to only achieving half the bandwidth as advertised, Howard said fibers often misbehaved or broke. "Once we got everything aligned and working, there were a lot of fiber attachment issues where they would just pop off because these are brittle fibers," Howard said. "We also saw, when we were doing the reflow process of the overall package, that we would run into optical problems due to thermal results, and we would end up having bad optical yields." To overcome these challenges Intel had to work with its partners to develop new materials with lower thermal barriers, Howard explained. * AMD on the edge: Stripped down Siena Epycs teased * Google sharpens AI toolset with new chips, GPUs, more at Cloud Next * Intel promises next year's Xeons will challenge AMD on memory, IO channels * Arm reveals just how vulnerable it is to trade war with China Tackling silicon bottlenecks This novel system was needed because while Intel's current commercial kit speeds through graph analytics workloads, it struggles to scale. "A Xeon can pretty much take one of the graph datasets and store it in its cache and just churn through it really quickly," said Intel's Howard. However, when you start to scale those datasets you start running into performance and efficiency bottlenecks. [networks] So, the team set out to develop a new kind of processor optimized for graph analytics and quickly discovered several quirks inherent to the workload that could be optimized for in silicon. "We immediately saw that they are massively parallel, in fact, embarrassingly parallel, so we could exploit a lot of parallelism to improve the overall performance," Howard said, adding that the team also uncovered issues with memory and cache utilization. "When we would bring a cache line into the processor, we would only usually use eight bytes of it and then end up throwing away the other 64 bytes without ever utilizing them," he said, explaining that this put unnecessary strain on the system any time there were a bunch of big out of order pipelines. A significant chunk in the center of the die is dedicated to routers which control the flow of data streaming in from the co-packaged optics. A significant chunk in the center of the die is dedicated to routers which control the flow of data streaming in from the co-packaged optics. - Click to enlarge These design considerations drove the team to develop this experimental processor, which TSMC builds using its 7nm FinFET process (in case you didn't know, Intel fabs a lot of non-CPU products at TSMC and has for years) and which features eight cores each with 66 threads. The chip also uses a novel memory architecture, an important inclusion to optimize it for graph analytics workloads, Howard explained. The chips are paired with 32GB of DDR5 4400MT/s memory which was accessed through a custom memory controller that allowed for 8-byte access granularity. According to Howard, this was advantageous as "anytime we're pulling a data line out of memory, we're going to try to utilize all of that data line as opposed to throwing away 7/8ths of it." PCIe 4.0 with an 8x connection allows for connectivity back to the host system. Intel also had to figure out a way to deal with the sheer volume of traffic coming in and out of the compute die, which in theory could be as much as 1TB/s. According to Howard, that requirement is the reason plenty of the die is dedicated to routers. And while you might think such a thread and network heavy chip might run hot, it doesn't. The chip tops out at 75 watts at 1GHz. According to Intel, a 16-sled configuration would handle 8TB of memory, 2,048 cores, 135,168 threads, and consume roughly 1.2kW of power. That's not bad considering that a single dual-socket Sapphire Rapids system with 112 cores and 224 threads could easily consume that much power under load. Intel imagines these chips could be built into rigs that scale to meshes encompassing 100,000 sleds and achieve near linear performance scaling for even the largest graph analytics workloads. However, this dream remains just that, as Intel has so far tested just two chips connected on its fabric. Whether Intel ever commercializes the design is going to depend on funding, Howard explained. "If people want to give us money, we are more than welcome to build these things out." (r) Get our Tech Resources # Share More about * Analytics * Fibre-optic * Intel More like these x More about * Analytics * Fibre-optic * Intel * Network * Semiconductor Narrower topics * Alder Lake * ASML * Black Hole * Broadband * Broadcom * Cadence Design Systems * Cellular network * Email * Ericsson * Ethernet * Firewall * Hot Chips * IETF * IPv4 * IPv6 * Loongson * Mobileye * Network interface card * Network switch * NXP Semiconductors * Pat Gelsinger * Radio Access Network * Router * Semiconductor Manufacturing International Corporation * Semiconductor Memory * SmartNIC * Software-defined network * STMicroelectronics * Streaming video * Submarine cable * Systems Approach * Tachyum * United Microelectronics Corporation * VPN * vPro * World Wide Web * x86 * Xen Broader topics * FPGA * Programming More about # Share 13 comment bubble on white COMMENTS More about * Analytics * Fibre-optic * Intel More like these x More about * Analytics * Fibre-optic * Intel * Network * Semiconductor Narrower topics * Alder Lake * ASML * Black Hole * Broadband * Broadcom * Cadence Design Systems * Cellular network * Email * Ericsson * Ethernet * Firewall * Hot Chips * IETF * IPv4 * IPv6 * Loongson * Mobileye * Network interface card * Network switch * NXP Semiconductors * Pat Gelsinger * Radio Access Network * Router * Semiconductor Manufacturing International Corporation * Semiconductor Memory * SmartNIC * Software-defined network * STMicroelectronics * Streaming video * Submarine cable * Systems Approach * Tachyum * United Microelectronics Corporation * VPN * vPro * World Wide Web * x86 * Xen Broader topics * FPGA * Programming TIP US OFF Send us news --------------------------------------------------------------------- Other stories you might like Beijing's silent treatment topples Tower Semiconductor merger with Intel Termination fee of $353 million wipes out Intel Foundry Services' revenue for last quarter Systems16 Aug 2023 | 8 Intel's Tower bid has shuffled off this mortal coil - so what about foundry plans? Analysis Maybe Pat dodged a bullet - mature process nodes aren't the kind of thing shareholders get excited about Systems18 Aug 2023 | 2 Intel promises next year's Xeons will challenge AMD on memory, IO channels Hot Chips Plus more insights on x86 titan's all E-core datacenter chips Systems28 Aug 2023 | 17 Paving an intentional road to a sustainable IT future For businesses that want to stay competitive, e-waste is an infelicitous fact of life - so what to do about it? Sponsored Feature [networks] US and China to keep talking about chip bans, just not when they'll end Intel and Micron made the agenda, but action to ease their woes did not Legal29 Aug 2023 | 2 AVX10: The benefits of AVX-512 without all the baggage Turns out bigger isn't always better Systems15 Aug 2023 | 14 China cooks covert chips, recruits global geeks to dodge US restrictions The Qiming is impeccable Systems24 Aug 2023 | 14 Getting meshy: BAE scores PS89m deal with MoD to build new battlefield network It'll replace the old one, also built by BAE, on a very tight schedule: The old junk's due to be ripped out in 2026 Networks25 Aug 2023 | 17 Get 'em while you can: Intel begins purging NUCs from inventory Is that a firesale we smell? Personal Tech14 Aug 2023 | 11 Intel seems to think Wi-Fi 7 is too cool for old-school Windows 10 What do you mean, you 'haven't updated yet'? Systems24 Aug 2023 | 52 Now Middle East nations banned from getting top-end Nvidia AI chips While ASML says it can keep selling DUV kit to China through 2023 Systems31 Aug 2023 | Rapidus ramps as construction begins on 2nm wafer fab Japanese foundry startup also shipping engineers off to US to study IBM chip tech Systems1 Sep 2023 | 2 The Register icon Biting the hand that feeds IT About Us* * Contact us * Advertise with us * Who we are Our Websites* * The Next Platform * DevClass * Blocks and Files Your Privacy* * Cookies Policy * Privacy Policy * T's & C's * Do not sell my personal information Situation Publishing Copyright. All rights reserved (c) 1998-2023 no-js