[HN Gopher] From Unemployment to Lisp: Running GPT-2 on a Teen's...
       ___________________________________________________________________
        
       From Unemployment to Lisp: Running GPT-2 on a Teen's Deep Learning
       Compiler
        
       Author : AymanB
       Score  : 79 points
       Date   : 2024-12-10 16:12 UTC (6 hours ago)
        
 (HTM) web link (github.com)
 (TXT) w3m dump (github.com)
        
       | AymanB wrote:
       | A couple months ago I found myself unemployed, uncertain about
       | what to do next. I wanted to learn more about deep learning, but
       | from a systems prespective. Coming from Andrew's Ng course on
       | supervised learning, I was eager to learn more about how deep
       | learning frameworks (or deep learning compilers) like Pytorch or
       | Tinygrad.
       | 
       | I started to poke around Tinygrad, learning from the tutorials I
       | found online, and I found it fascinating because it was an actual
       | compiler, it took conventional python code and translated them
       | into an Abstract Syntax Tree that was parsed into UOps and
       | ScheduleItems, to finally have a codegen layer. While the design
       | was interesting, the code was hard to read.
       | 
       | That's when I stumbled across something completly unexpected, A
       | deep learning compiler built on Common Lisp, maintained by a
       | Japanese 18-year-old during his gap year. And currently we have
       | acomplished something great, it can run gpt2!
       | 
       | For now, it just generates C-kernels, but in the future we would
       | like to support cuda codegen as well as many other features, and
       | serve as a learning tool for anyone who would like to get to work
       | on deep learning compilers in Common Lisp.
        
         | a2code wrote:
         | In general, a compiler takes source code and generates object
         | code or an executable. Can you elaborate on what your compiler
         | takes as input and generates as an output?
        
       ___________________________________________________________________
       (page generated 2024-12-10 23:00 UTC)