[HN Gopher] Solar-10B
       ___________________________________________________________________
        
       Solar-10B
        
       Author : tosh
       Score  : 16 points
       Date   : 2023-12-13 20:53 UTC (2 hours ago)
        
 (HTM) web link (twitter.com)
 (TXT) w3m dump (twitter.com)
        
       | filterfiber wrote:
       | I know the hugging face leaderboard isn't wildly accurate.
       | 
       | But the top models right now are almost all under 70B. Most are
       | 7B, and the top is 10B. If the benchmarks are even remotely
       | accurate then this is rather wild.
       | 
       | Apparently multiple groups found different "secret sauces", names
       | upstage and whatever UNA is?
        
         | appplication wrote:
         | I mean this isn't too surprising that smaller models do better.
         | I imagine transformers are as prone to overfitting as any
         | statistical data model. Also there is probably some selection
         | bias: bigger models are more expensive and there are just less
         | people training and iterating with them
        
       ___________________________________________________________________
       (page generated 2023-12-13 23:01 UTC)