[HN Gopher] Are language models good at making predictions?
       ___________________________________________________________________
        
       Are language models good at making predictions?
        
       Author : RationalDino
       Score  : 5 points
       Date   : 2023-11-07 21:50 UTC (1 hours ago)
        
 (HTM) web link (www.lesswrong.com)
 (TXT) w3m dump (www.lesswrong.com)
        
       | pk-protect-ai wrote:
       | They are excellent predictors of what they have been taught to
       | predict: composing text in different languages. They are so
       | proficient at these predictions that it is challenging to
       | distinguish LLMs text from human-written text. As a byproduct of
       | learning from Common Crawl, they are adept at predicting average
       | "internet-human" behavior, leading to phenomena such as "Sydney".
       | Their predictive capabilities are so impressive that you can play
       | chess with them and find a formidable opponent. Despite never
       | having seen anything beyond the text in their training data, they
       | demonstrate spatial comprehension. They are universal
       | approximators...
        
         | RationalDino wrote:
         | Yes. The fact that they are so good at these things is why we
         | might reasonably ask how good they are at making predictions
         | about the world.
         | 
         | The fact that their predictions are not particularly good is
         | therefore of interest.
        
       | IanCal wrote:
       | It feels a shame to do this and cram it all into one response,
       | formatted in JSON. That doesn't really give them much room to
       | think.
        
       ___________________________________________________________________
       (page generated 2023-11-07 23:00 UTC)