[HN Gopher] How Fast Do Algorithms Improve? [pdf]
       ___________________________________________________________________
        
       How Fast Do Algorithms Improve? [pdf]
        
       Author : djoldman
       Score  : 12 points
       Date   : 2021-09-22 19:59 UTC (1 days ago)
        
 (HTM) web link (ieeexplore.ieee.org)
 (TXT) w3m dump (ieeexplore.ieee.org)
        
       | tedsanders wrote:
       | Progress in computation can be decomposed into three factors:
       | 
       | Perf = (perf / op) * (op / $) * $
       | 
       | (1) Perf / op is essentially software progress, the subject of
       | this article. (And for AI/ML, you can further decompose it into
       | algorithm progress and training data progress.)
       | 
       | (2) Op / $ is essentially Moore's law, which has been slowing
       | down and can be further decomposed into: Ops / $ = (ops /
       | transistor) * (transistor / area) * (area / $)
       | 
       | (3) $ is willingness to spend money. This has probably been the
       | biggest source of progress for AI projects in the past 10 years,
       | with Deep Mind and Open AI and Google Brain and Silicon Valley ad
       | tech spending billions to build models more complex and capable
       | than any that have come before. http://www.rossgritz.com/trends-
       | in-deepmind-operating-costs/...
        
       ___________________________________________________________________
       (page generated 2021-09-23 23:00 UTC)