[HN Gopher] Transformer models: an introduction and catalog
       ___________________________________________________________________
        
       Transformer models: an introduction and catalog
        
       Author : mariuz
       Score  : 64 points
       Date   : 2023-02-16 08:01 UTC (14 hours ago)
        
 (HTM) web link (arxiv.org)
 (TXT) w3m dump (arxiv.org)
        
       | abc20230215 wrote:
       | Does not even list Lundahl transformers...
        
         | [deleted]
        
       | adamnemecek wrote:
       | I have recently written a paper on understanding transformer
       | learning via the lens of coinduction & Hopf algebra.
       | 
       | https://arxiv.org/abs/2302.01834v1
       | 
       | The learning mechanism of transformer models was poorly
       | understood however it turns out that a transformer is like a
       | circuit with a feedback.
       | 
       | I argue that autodiff can be replaced with what I call in the
       | paper Hopf coherence.
       | 
       | Furthermore, if we view transformers as Hopf algebras, one can
       | bring convolutional models, diffusion models and transformers
       | under a single umbrella.
       | 
       | I'm working on a next gen Hopf algebra based machine learning
       | framework.
       | 
       | Join my discord if you want to discuss this further
       | https://discord.gg/mr9TAhpyBW
        
         | erichocean wrote:
         | > _Furthermore, if we view transformers as Hopf algebras, one
         | can bring convolutional models, diffusion models and
         | transformers under a single umbrella._
         | 
         | Have you written any more about this?
        
           | adamnemecek wrote:
           | Look into the connection between diffusion and Hopf algebras.
        
       | theredlancer wrote:
       | Where's Cliffjumper and Ironside?
        
         | [deleted]
        
         | zndr wrote:
         | I'm glad I'm not the only one looking for a taxonomy of
         | refugees from the great Cybertron wars
        
       | swyx wrote:
       | figure 5 on page 10 is a ridiculously small font and unreadable.
       | i wish there was a better way to display this kind of info on
       | PDFs
        
         | dylan604 wrote:
         | From the bottom of the page in question: "Figure 5: You can
         | access the original table at
         | https://docs.google.com/spreadsheets/d/
         | 1ltyrAB6BL29cOv2fSpNQnnq2vbX8UrHl47d7FkIf6t4 for easier
         | browsing across the different model features."
        
         | mdp2021 wrote:
         | > _i wish there was a better way to display this kind of info
         | on PDFs_
         | 
         | ...Portable Document Format was  /born/ to display vector (i.e.
         | you just zoom in)... The error in the page was to embed a
         | raster image of text!
        
         | [deleted]
        
       | peresthe wrote:
       | "The goal of this paper is to offer a somewhat comprehensive but
       | simple catalog and classification of the most popular Transformer
       | models."
       | 
       | Yet of the 6 comments here, 2 of them are complaining about
       | missing models and three more are arguing about the typesetting
       | on figures.
        
       ___________________________________________________________________
       (page generated 2023-02-16 23:00 UTC)