[HN Gopher] OpenTPU: Open-Source Reimplementation of Google Tens...
___________________________________________________________________
OpenTPU: Open-Source Reimplementation of Google Tensor Processing
Unit (TPU)
Author : walterbell
Score : 152 points
Date : 2025-05-27 23:10 UTC (23 hours ago)
(HTM) web link (github.com)
(TXT) w3m dump (github.com)
| mdaniel wrote:
| Yeowzers that FAQ is filled with watch-outs
|
| The /forks contained https://github.com/csirlin/OpenTGPTPU which
| had a commit 3 hours ago but it seems they have not yet updated
| the FAQ for their version. Anyway, the fact it has commits
| greater than 8 years ago makes it seem like a more reasonable
| submission
| walterbell wrote:
| Google TPU engineers used open-source Chisel for ASIC design
| (2018), https://youtube.com/watch?v=x85342Cny8c
|
| _" Google Edge TPU devices"_, 100 comments (2019),
| https://news.ycombinator.com/item?id=19130896 &
| https://news.ycombinator.com/item?id=19313813
|
| _" Coral Edge TPU review"_, 100 comments (2020),
| https://news.ycombinator.com/item?id=24808755
|
| _" TPU transformation: 10 years of our AI-specialized chips"_,
| 60 comments (2024), https://news.ycombinator.com/item?id=41148532
| dekhn wrote:
| The site confuses the inference engine in the Edge TPU with the
| datacenter TPU. They are two unrelated projects. Based on the
| paper they're borrowing from, I think they are trying to go for
| a much older datacenter inference-only TPU, or only
| implementing the inference capabilities of the datacenter TPU.
| walterbell wrote:
| Are there recent papers on datacenter TPU?
| dekhn wrote:
| Yes.
| walterbell wrote:
| David Patterson overview (2023),
| https://www.cs.ucla.edu/wp-
| content/uploads/cs/PATTERSON-10-L...
|
| TPU v4 (2023), https://arxiv.org/abs/2304.01433
| flakiness wrote:
| [2017] (https://arxiv.org/abs/1704.04760)
| walterbell wrote:
| [May 2025]
| (https://github.com/csirlin/OpenTGPTPU/commits/master)
| flakiness wrote:
| Wow they have kept working on this! Thanks for pointing this!
| very impressive.
| whimsicalism wrote:
| > The TPU is Google's custom ASIC for accelerating the inference
| phase of neural network computations.
|
| this seems hopelessly out of date/confused
| walterbell wrote:
| Additional text from Google's 2017 paper abstract says:
| This paper evaluates a custom ASIC---called a Tensor Processing
| Unit (TPU)---deployed in datacenters since 2015 that
| accelerates the inference phase of neural networks (NN). The
| heart of the TPU is a 65,536 8-bit MAC matrix multiply unit
| that offers a peak throughput of 92 TeraOps/second (TOPS) and a
| large (28 MiB) software-managed on-chip memory. The
| TPU's deterministic execution model is a better match to the
| 99th-percentile response-time requirement of our NN
| applications than are the time-varying optimizations of CPUs
| and GPUs (caches, out-of-order execution, multithreading,
| multiprocessing, prefetching, ...) that help average throughput
| more than guaranteed latency. The lack of such
| features helps explain why, despite having myriad MACs and a
| big memory, the TPU is relatively small and low power. We
| compare the TPU to a server-class Intel Haswell CPU and an
| Nvidia K80 GPU, which are contemporaries deployed in the same
| datacenters.
| whimsicalism wrote:
| hence the out of date part of my comment
| walterbell wrote:
| Recent (2024) description by Google,
| https://cloud.google.com/blog/transform/ai-specialized-
| chips... TPUs were purpose-built
| specifically for AI. TPUs are an application-specific
| integrated circuit (ASIC), a chip designed for a single,
| specific purpose: running the unique matrix and vector-
| based mathematics that's needed for building and running AI
| models.. TPU v2.. built an interconnected
| machine -- our first TPU pod -- with 256 TPU chips
| connected with a very high-bandwidth, custom interconnect..
| liquid cooling was added with TPU v3 to help address
| efficiency needs, while TPU v4 introduced optical circuit
| switches to allow the chips in pods to communicate even
| faster and more reliably. TPUs also underpin
| Google DeepMind's cutting-edge foundation models, including
| the newly unveiled Gemini 1.5 Flash, Imagen 3, and Gemma 2,
| propelling advancements in AI.. Forget about a single chip,
| or a single TPU pod -- we're building a global network of
| data centers filled with TPUs.
| throwawaymaths wrote:
| what's the memory bandwidth? IIRC that is the limiting factor
| in LLM hardware today
| walterbell wrote:
| Slide 21, https://files.futurememorystorage.com/proceedings
| /2024/20240... TPUv3 TPUv4
| HBM2 BW 900 GB/s 1200 GB/s
| surfmike wrote:
| How would you describe it instead? Curious and learning
| imtringued wrote:
| Google does everything, both inference and training, on their
| TPUs.
|
| Inference is easier, since the person deploying a model knows
| the architecture ahead of time and therefore can write custom
| code for their particular model.
|
| When training you want to be as flexible as possible. The
| framework and hardware should not impose any particular
| architecture. This means lots of kernels and combinations of
| kernels. Miss one and you're out.
| throwawaymaths wrote:
| > Miss one and you're out.
|
| well these days since everything is transformer, your pool
| of choices is less daunting and theres only about four or
| five places that someone might get clever.
| dgacmu wrote:
| They're not confused at all, this is just a (correct)
| description of TPU v1. The repository is 8 years old.
| andutu wrote:
| There is an excellent paper and talk on how Google's TPU cluster
| is managed:
| https://www.usenix.org/conference/nsdi24/presentation/zu.
| westurner wrote:
| Can [OpenTPU] TPUs be fabricated out of graphene, with
| nanoimprinting or a more efficient approach?
|
| From https://news.ycombinator.com/item?id=42314333 :
|
| >> _From "A carbon-nanotube-based tensor processing unit" (2024)
| https://www.nature.com/articles/s41928-024-01211-2 :_
|
| >>> _Using system-level simulations, we estimate that an 8 bit
| TPU made with nanotube transistors at a 180 nm technology node
| could reach a main frequency of 850 MHz and an energy efficiency
| of 1 tera-operations per second per watt._
| westurner wrote:
| What about QPUs though?
|
| Can QPUs (Quantum Processing Units) built on with electrons in
| superconducting graphene ever be faster than photons in
| integrated nanophotonics?
|
| There are integrated parametric single-photon emitters and
| detectors.
|
| Is there a lower cost integrated nanophotonic coherent light
| source for [quantum] computing than a thin metal wire?
|
| "Electrons turn piece of wire into laser-like light source"
| (2022) https://news.ycombinator.com/item?id=33493885
___________________________________________________________________
(page generated 2025-05-28 23:01 UTC)