[HN Gopher] Transformer Inference Arithmetic (2022)
       ___________________________________________________________________
        
       Transformer Inference Arithmetic (2022)
        
       Author : lawrencechen
       Score  : 41 points
       Date   : 2023-05-13 08:17 UTC (2 days ago)
        
 (HTM) web link (kipp.ly)
 (TXT) w3m dump (kipp.ly)
        
       | thatcherc wrote:
       | Author seems to be using billion == 10^12 instead of the common
       | billion == 10^9. A lot of the math still works out since there's
       | a multiply and a divide by a billion, but it is a little
       | confusing to see passages like this:
       | 
       | > Given the parameter count, we can multiply by two to get bytes.
       | So to calculate the size of the weights for a 52B model.
       | 
       | > 52e12[?]2 = 104e12 bytes [?] 104GB
        
         | pulvinar wrote:
         | I assume that's a slip back to the old British billion. But
         | luckily that mistake doesn't affect that result, and does get
         | it right in the next equation:
         | 
         | > 2,097,152 bytes [?] 0.002GB
        
         | kipply wrote:
         | oops will fix. never using names of numbers of zeroes ever
         | again. that stuff sucks, i will express everything in
         | scientific notation
        
       | gxh8N wrote:
       | Very nicely written. I also like how it changes color every time
       | I reload the article.
        
       ___________________________________________________________________
       (page generated 2023-05-15 23:02 UTC)