[HN Gopher] Discovering Novel Algorithms with AlphaTensor
       ___________________________________________________________________
        
       Discovering Novel Algorithms with AlphaTensor
        
       Author : alexrustic
       Score  : 74 points
       Date   : 2022-10-05 15:18 UTC (7 hours ago)
        
 (HTM) web link (www.deepmind.com)
 (TXT) w3m dump (www.deepmind.com)
        
       | MikeYasnev007 wrote:
        
       | homarp wrote:
       | "Since 1969 Strassen's algorithm has famously stood as the
       | fastest way to multiply 2 matrices - but with AlphaTensor.
       | Deepmind has found a new algorithm that's faster, with potential
       | to improve efficiency by 10-20% across trillions of calculations
       | per day!"
        
         | klyrs wrote:
         | shhhh don't tell Coppersmith... or Alman&Williams
        
         | xdavidliu wrote:
         | Strassen has _not_ stood as the fastest way, whether you look
         | at the exponent or the constant factor.
         | 
         | It's probably more accurate to say Strassen was the first in a
         | series of complexity improvements to matrix multiplication,
         | most of whom are theoretical curiosities and have prohibitively
         | high constant factors preventing them from ever been used in
         | practice.
         | 
         | Even if all you cared about was the exponent, Strassen was
         | beaten decades ago.
         | 
         | https://en.wikipedia.org/wiki/Matrix_multiplication#Computat...
        
       | homarp wrote:
       | see https://news.ycombinator.com/item?id=33096580 for discussion
       | of the Nature paper
        
       | dougmwne wrote:
       | This is amazing. It sounds like the summary is that they turned
       | matrix multiplication of various sized matrices into a kind of
       | game, then had AlphaZero, the game playing AI, play the game and
       | find optimal strategies. The new optimal strategies for "winning"
       | the game in less moves were actually new matrix multiplication
       | algorithms, some of which had been discovered previously, some of
       | which were new. Many thousands of algorithms were found across a
       | range of matrix sizes.
       | 
       | And of course aside from the intellectual curiosity, our GPUs are
       | mega matrix multiplication machines and these new algorithms
       | could be tuned to specific hardware and improve the overall
       | graphics and ML performance by 10-20% on existing hardware.
       | 
       | Cool!
        
       | buildbot wrote:
       | I wonder if this technique could be used to find better/more
       | mathematically efficient self attention algorithms for
       | transformers in general. Meta programming with AI...
        
       ___________________________________________________________________
       (page generated 2022-10-05 23:01 UTC)