[HN Gopher] Looking Back at Speculative Decoding
       ___________________________________________________________________
        
       Looking Back at Speculative Decoding
        
       Author : veryluckyxyz
       Score  : 19 points
       Date   : 2025-03-01 06:24 UTC (2 days ago)
        
 (HTM) web link (research.google)
 (TXT) w3m dump (research.google)
        
       | veryluckyxyz wrote:
       | https://pytorch.org/blog/hitchhikers-guide-speculative-decod...
       | 
       | https://colab.research.google.com/github/sanchit-gandhi/note...
        
       | numeri wrote:
       | I've been slightly annoyed by how the Speculative Decoding paper
       | has gotten all the credit for the technique - I first learned
       | about the technique from a paper more than a year older[1],
       | Shallow Aggressive Decoding.
       | 
       | They introduce the same method, but apply it to grammatical error
       | correction, meaning the "draft" output is just the input itself.
       | The Speculative Decoding paper tries to emphasize differences
       | between this and their method, saying that theirs is more
       | general, as they apply it to more domains, allowing the draft to
       | come from a smaller model, and extend it to allow sampling.
       | 
       | All of that is great, and deserves another paper, but doesn't
       | deserve the credit for inventing and rights to rename the method,
       | especially when they were aware of Shallow Aggressive Decoding
       | before uploading their first draft.
       | 
       | [1]: https://arxiv.org/abs/2106.04970
        
       ___________________________________________________________________
       (page generated 2025-03-03 23:01 UTC)