[HN Gopher] How to Write a Fast Matrix Multiplication from Scrat...
       ___________________________________________________________________
        
       How to Write a Fast Matrix Multiplication from Scratch with Tensor
       Cores (2024)
        
       Author : skidrow
       Score  : 87 points
       Date   : 2025-04-19 14:42 UTC (8 hours ago)
        
 (HTM) web link (alexarmbr.github.io)
 (TXT) w3m dump (alexarmbr.github.io)
        
       | light_hue_1 wrote:
       | Closely related to this if you're interested in the topic if Deep
       | Mind's guide on how to scale your model.
       | 
       | https://jax-ml.github.io/scaling-book/roofline/
        
       | ashvardanian wrote:
       | Great article -- and several other high-quality deep dives linked
       | at the end! Here's another one on the H100 that I found
       | particularly useful:
       | <https://cudaforfun.substack.com/p/outperforming-cublas-on-h1...>
       | 
       | I agree with the author that programming GEMM on newer GPUs is a
       | very different experience, though I'm wondering if "newer GPUs
       | are [actually strictly] better"? It seems like there should still
       | be some highly cost-effective use cases for T4 GPUs -- aren't
       | there?
        
       | westurner wrote:
       | Multiplication algorithm:
       | https://en.wikipedia.org/wiki/Multiplication_algorithm
       | 
       | From https://news.ycombinator.com/item?id=40519828 re: LLMs and
       | matrix multiplication with tensors:
       | 
       | > _" You Need to Pay Better Attention" (2024)
       | https://arxiv.org/abs/2403.01643 :_
       | 
       | >> _Our first contribution is Optimised Attention, which performs
       | similarly to standard attention, but has 3 /4 as many parameters_
       | and one matrix multiplication fewer per head. _Next, we introduce
       | Efficient Attention, which performs on par with standard
       | attention with only 1 /2 as many parameters as many parameters_
       | and two matrix multiplications fewer per head _and is up to twice
       | as fast as standard attention. Lastly, we introduce Super
       | Attention, which surpasses standard attention by a significant
       | margin in both vision and natural language processing tasks while
       | having fewer parameters and matrix multiplications._
       | 
       | From "Transformer is a holographic associative memory" (2025)
       | https://news.ycombinator.com/item?id=43029899 ..
       | https://westurner.github.io/hnlog/#story-43028710 :
       | 
       | >>> _Convolution is in fact multiplication in Fourier space (this
       | is the convolution theorem [1]) which says that Fourier
       | transforms convert convolutions to products._
       | 
       | From https://news.ycombinator.com/item?id=41322088 :
       | 
       | > _" A carbon-nanotube-based tensor processing unit" (2024)_
        
       | astee wrote:
       | If I ever need a fast matmul, you're hired.
        
       | gradascent wrote:
       | Great deep dive. I've learned a lot already and haven't even
       | finished the introduction
        
       | atum47 wrote:
       | In my parallel programming class we used several techniques to
       | increase the speed of matrix multiplication, and compared them. I
       | vaguely remember using OpenMP and cuda. I need to look into my
       | backups to see if I still have those codes. Specially the cuda
       | one, I wonder how similar it is to tensors
        
       ___________________________________________________________________
       (page generated 2025-04-19 23:00 UTC)