[HN Gopher] SRAM in AI: The Future of Memory
       ___________________________________________________________________
        
       SRAM in AI: The Future of Memory
        
       Author : PaulHoule
       Score  : 35 points
       Date   : 2023-11-13 07:11 UTC (15 hours ago)
        
 (HTM) web link (semiengineering.com)
 (TXT) w3m dump (semiengineering.com)
        
       | thomasfl wrote:
       | I clicked on this because my bicycle has SRAM gear shifters.
       | Disappointed to see an article about computers.
        
         | kgabis wrote:
         | It's even more confusing for polish speaking folks.
        
           | sebastiennight wrote:
           | Co oznacza slowo "sram" w jezyku polskim? DeepL nie daje mi
           | zadnych trafien
        
             | Cockbrand wrote:
             | Google Translate to the rescue: https://translate.google.co
             | m/?sl=pl&tl=en&text=sram&op=trans...
        
               | Gravityloss wrote:
               | I guess the bicycle parts company was founded before
               | internet...
        
               | thomasfl wrote:
               | SRAM bike parts was founded in 1985.
        
             | codebastard wrote:
             | https://cooljugator.com/pl/sra%C4%87
        
               | sebastiennight wrote:
               | That's hilarious. Thanks for the help :)
        
       | neilmovva wrote:
       | I can't universally agree with the headline statement. The
       | article focuses on the pros of SRAM, which are real -- peak
       | bandwidth (e.g. 5 TB/s out of the H100's L2) and lower energy per
       | bit transferred (the rule of thumb I remember is ~10x lower than
       | HBM).
       | 
       | But the companies who already bet big on SRAM in AI, Cerebras in
       | particular and Graphcore to lesser extent, aren't obviously
       | running away with the AI performance crown. Seems like LLMs need
       | more memory capacity than anyone expected, to the point where
       | even HBM stacks somewhat limit the scale of current models. Maybe
       | the next version of Cerebras WSE can get closer to 100GB of on-
       | chip memory, and serve some useful LLMs very efficiently -
       | excited to see what they can do with more modern processes!
       | 
       | I think innovation in SRAM packaging, like AMD's stacking in "3D
       | V-Cache", is also promising and might play a larger role in AI
       | accelerators going forward. But it's important to note that while
       | both are SRAM, performance of AMD's stacked L3 is not yet
       | comparable to a GPU's centralized L2 - it's more like HBM today
       | in latency and bandwidth.
        
       ___________________________________________________________________
       (page generated 2023-11-13 23:02 UTC)