[HN Gopher] llm.c: multi-GPU, bfloat16, flash attention, ~7% fas...
       ___________________________________________________________________
        
       llm.c: multi-GPU, bfloat16, flash attention, ~7% faster than
       PyTorch
        
       Author : tosh
       Score  : 104 points
       Date   : 2024-05-03 18:24 UTC (1 days ago)
        
 (HTM) web link (twitter.com)
 (TXT) w3m dump (twitter.com)
        
       | gpapilion wrote:
       | I'd be happier with 93% of PyTorch but works on multiple gpu
       | manufacturers.
        
         | reallymental wrote:
         | That... wasn't the original intention of the project. It was to
         | create a C version of the PyTorch code that could train GPT-2.
        
         | tyfighter wrote:
         | Yeah, I'm sure that's what anyone trying to build some kind of
         | AI startup that's managed to acquire a small handful of A100 or
         | even better H100s thinks too. "Those cards sure were expensive,
         | but ethically, I'd rather the software run slower to give me
         | future imaginary options than to get the most out the hardware
         | I just bought."
        
       | ein0p wrote:
       | Crated over the period of like 4 weeks by random people all over
       | the internet
        
       | pama wrote:
       | Much faster yet than stable pytorch 2.3 (46% on A100, as per the
       | tweet), and much much faster yet compared to pytorch 2.2, which
       | was the stable version a couple weeks ago. Also llm.c is much
       | faster yet when the performance comparison is on H100 instead of
       | A100, or on multiple GPU instead of a single one.
        
       | michaelgiba wrote:
       | it's pretty impressive that PyTorch is only 7% slower than this
       | given it can be used so generally
        
       ___________________________________________________________________
       (page generated 2024-05-04 23:01 UTC)