[HN Gopher] Forcing Flash Attention onto a TPU and Learning the ...
       ___________________________________________________________________
        
       Forcing Flash Attention onto a TPU and Learning the Hard Way
        
       Author : azhng
       Score  : 26 points
       Date   : 2026-03-08 03:57 UTC (4 days ago)
        
 (HTM) web link (archerzhang.me)
 (TXT) w3m dump (archerzhang.me)
        
       | gdiamos wrote:
       | One of my lessons in using different accelerators, whether they
       | be different NVIDIA versions, or GPU->TPU, etc is that someone
       | needs to do this work of indexing, partitioning, mapping,
       | scheduling, and benchmarking. That work is labor intensive.
       | 
       | In this case, google has already done it, and that will be true
       | for high resourced accelerator companies like Google working with
       | the most popular operations like attention.
       | 
       | As long as you use those operations, you are okay. But if you do
       | something different, you need to be prepared to do all of this
       | yourself.
        
       | refulgentis wrote:
       | It broke my heart to have a visceral "I'm being slop'd" reaction
       | reading this: it's _such_ good work, and AI 's barely used
       | AFAICT, but there's enough odd transitions and copy-pasta'd
       | markdown that you get the subconcious "this is AI" reaction
       | regardless.
       | 
       | Many sentences are 3x as long as it normally would be in subtle
       | ways (to wit: "My flash attention is 35x slower than the fused
       | standard at n=4096. Not a little worse. Catastrophically
       | worse."), it really wears on attention. (pun intended) It brings
       | literary voice to a technical blog post, and a very difficult
       | process-oriented technical blog post. I have to reallocate my
       | unfortunately-limited brain cells from "maintaining state of
       | where we are in the process" to "is this cutesy fluff or
       | important" and I've never had to do that in 37 years with
       | technical blog posts.
       | 
       | The Markdown gets _bad_. Bolding is used for important phrases
       | (like a human would), then, all of a sudden, after the  "Inside a
       | TPU chip" header its being used every other sentence, on anything
       | that is a proper noun/would have a Wikipedia article. It got so
       | weird that at some point I was like "a human _definitely_ didn 't
       | let this through...they must be links?" and tried clicking them.
       | 
       | It's doubly bad at that point, because markdown tables start
       | coming in hot and heavy too. So you're left with "It's pretty
       | apparent the LLM did it from here, and I can't keep trying to
       | keep the state of the process in my head while trying to figure
       | out if the bolding is important, _reflexive close tab_
        
       ___________________________________________________________________
       (page generated 2026-03-12 23:00 UTC)