[HN Gopher] Logit Prisms: Decomposing Transformer Outputs for Me...
       ___________________________________________________________________
        
       Logit Prisms: Decomposing Transformer Outputs for Mechanistic
       Interpretability
        
       Author : xcodevn
       Score  : 38 points
       Date   : 2024-06-17 11:57 UTC (2 days ago)
        
 (HTM) web link (neuralblog.github.io)
 (TXT) w3m dump (neuralblog.github.io)
        
       | behnamoh wrote:
       | Off topic: can anyone tell me what the author has used to convert
       | their paper into this beautiful webpage? It's apparently a trend
       | with ML papers to have their own website.
        
         | nbgoodall wrote:
         | From the source, looks like they're using Quarto:
         | https://quarto.org
        
           | behnamoh wrote:
           | Thank you!
        
           | esafak wrote:
           | A popular alternative is https://distill.pub/guide/
        
       | xcodevn wrote:
       | If you're reading this, you may be interested in my other work on
       | "Exploring Llama-3 MLP Neurons" as well.
       | 
       | https://neuralblog.github.io/llama3-neurons/
        
       | nico wrote:
       | Tangent, the screen reader recording is cool. The main issue for
       | me is that it doesn't know where I am on the page, so it just
       | starts from the beginning and I need to guess the position of the
       | relevant audio based on how far down on the page I am. It would
       | be cool to have some sort of bookmarks that match both the text
       | and the audio, so that I can skip both to the same section at the
       | same time
        
       | XorNot wrote:
       | Okay so what stood out to me here is the math work: namely, if we
       | can identify the parts of the model which are doing the math...
       | Can that be "augmented" to generalise it? Could you take over the
       | math neurons and either optimise them by hand or plug in a
       | software "implant" to do the work properly?
        
         | Lerc wrote:
         | It will be an area of active research, but it remains to be
         | seen if it is overall beneficial.
         | 
         | Custom architecture components break the homogeneity that
         | enables the massive parallelism.
         | 
         | There may be an approach similar to some FPGAs where there are
         | a few things that are commonly needed that might be implemented
         | specifically to avoid constructing them out of generalisable
         | parts. The calculation is quite different though because
         | hardware has more fixed limits and being fixed once
         | manufactured means a great deal more forethought has to take
         | place. Software models, being more flexible can avoid the extra
         | thinking by adding a few million parameters.
         | 
         | One possibility is KANs which can also provide a calculable
         | expression. I could see a fancy convolution KAN being converted
         | into an expression and then applied across an entire input.
         | 
         | I feel like maybe in the future there may be a compiler that
         | can make custom GPU code for specific pathways.
         | 
         | In the short term it might be useful if there is some found
         | formula that requires many layers that can be calculated
         | earlier with a custom expression and fed directly into a higher
         | layer enabling the information to be more broadly used.
        
       ___________________________________________________________________
       (page generated 2024-06-19 23:01 UTC)