[HN Gopher] Research papers on ML in Compilers
       ___________________________________________________________________
        
       Research papers on ML in Compilers
        
       Author : snprajwal
       Score  : 87 points
       Date   : 2023-06-21 13:01 UTC (10 hours ago)
        
 (HTM) web link (github.com)
 (TXT) w3m dump (github.com)
        
       | StackOverlord wrote:
       | Related C2 wik page:
       | 
       | https://wiki.c2.com/?SufficientlySmartCompiler
        
       | dzign wrote:
       | It's interesting to add the work about Deepmind AlphaDev to this
       | list.
        
       | achr2 wrote:
       | I know it is irrational - as compilers already perform
       | optimizations that are non intuitive - but my lizard brain really
       | doesn't like the idea of ML possibly changing semantics at a
       | layer that is unobservable.
        
         | bilekas wrote:
         | Yeah, funny enough I was reading the "coding machines" short
         | story yesterday after someone linked it and I had a bit of a
         | giggle imagining the sentient compiler here.
        
         | UncleMeat wrote:
         | This sort of work isn't going "i dunno, emit some new
         | instructions based on a model and hope it is correct." ML
         | techniques fall into one of two categories:
         | 
         | 1. Decisions that have no semantic change in the program but
         | affect performance. This is things like code layout or register
         | allocation that will be more cache friendly. This is what I
         | expect to show up in industrial optimizing compilers more and
         | more over the next bunch of years.
         | 
         | 2. Decisions where you can prove that the new code sequence is
         | semantically identical to the original code sequence via some
         | formal technique. This is basically the same as various
         | superoptimization techniques but with a new search strategy and
         | therefore no new concerns about correctness. I'd expect this to
         | remain niche and only used offline to generate particularly
         | optimized sequences in extremely hot and small code paths.
        
       | hoosieree wrote:
       | My research (ML for binary function recognition) uses obfuscation
       | and diverse compilation for data augmentation.
       | 
       | Intuitively, obfuscation is a form of anti-optimization. My hope
       | is that it's differentiable, so perhaps given enough knowledge of
       | obfuscation, an ML model can make corresponding optimizations to
       | "undo" them, then for more performance, apply those same un-
       | obfuscations to code that _hasn 't_ already been obfuscated.
        
         | carom wrote:
         | I think this is a great line of research as it may solve the
         | reasoning portion of AGI. The difficulty is that you would want
         | to prove equivalence. LLMs can output stuff but for programs to
         | execute you need to output _correct_ stuff.
        
       | lapinot wrote:
       | I'd be more interested in the reverse, that is compiler and PL
       | techniques applied to machine learning.
        
         | anlunx wrote:
         | You might be interested in this:
         | https://github.com/merrymercy/awesome-tensor-compilers
        
       | sjkoelle wrote:
       | There is a mathematical description of programming compilers I
       | read once (think it was a survey paper) that I found very clear
       | but I've never been able to find since. Basically defined a
       | compiler as a type of function. Any good suggestions?
        
         | noblethrasher wrote:
         | Perhaps this one from Graham Hutton?
         | 
         | "A parser for things
         | 
         | Is a function from strings
         | 
         | To lists of pairs
         | 
         | Of things and strings"
        
           | sjkoelle wrote:
           | hmmmm maybe something like page 184 here
           | https://apps.dtic.mil/sti/pdfs/ADA327435.pdf
        
           | AugustoCAS wrote:
           | Thanks for sharing this. I became a bit of a fan of Dr Hutton
           | after I followed his Functional Programming in Haskell
           | classes on Youtube.
        
       | hetzenmat wrote:
       | Note that this refers to ML as in machine learning and not the ML
       | language family, as I hoped.
        
         | choeger wrote:
         | Me too :(.
        
         | brookst wrote:
         | There are so many flavors of machine learning that we should
         | probably just refer to whole field as XML.
        
           | TeMPOraL wrote:
           | At some point we might need to cross-reference (X-ref)
           | language models. This may involve using a dedicated data
           | interchange format - perhaps an extensible markup language
           | for language model cross-referencing. Or, in short, LMX-XML.
        
         | Findecanor wrote:
         | My mind still unabbreviates "ML" into "Machine Language" before
         | anything else, only because it was the first of these three for
         | me.
        
         | devsegal wrote:
         | What a cool language!
        
         | [deleted]
        
       ___________________________________________________________________
       (page generated 2023-06-21 23:02 UTC)