Post AntXsXWH9dUnMAipEG by garymarcus@sigmoid.social
 (DIR) More posts by garymarcus@sigmoid.social
 (DIR) Post #AntXsXWH9dUnMAipEG by garymarcus@sigmoid.social
       2024-11-09T23:52:32Z
       
       0 likes, 0 repeats
       
       Game over, I won. GPT is hitting a period of dimishing returns, just like I said it would.Analysis and breaking news here: https://open.substack.com/pub/garymarcus/p/confirmed-llms-have-indeed-reached
       
 (DIR) Post #AntXsYra9u8JWXhM4u by Erklaerbaer@mastodon.social
       2024-11-10T07:20:05Z
       
       0 likes, 0 repeats
       
       @garymarcus and here is the link https://www.newscientist.com/article/2353751-ai-chatbots-could-hit-a-ceiling-after-2026-as-training-data-runs-dry/
       
 (DIR) Post #AntXsaBpE7v5dcB2Gm by futurebird@sauropods.win
       2024-11-10T09:04:22Z
       
       1 likes, 2 repeats
       
       @Erklaerbaer @garymarcus Maybe it's time to think about why the "learning" algorithms can't get more out of the data? A child learns to talk by listening to people around them talk. They do not need to hear billions of sentences. They do not consume vast amounts of energy either.
       
 (DIR) Post #AntYcUtVFPtwXEV6TQ by Erklaerbaer@mastodon.social
       2024-11-10T09:12:41Z
       
       0 likes, 0 repeats
       
       @futurebird @garymarcus there is an awesome podcasts series of the Santa Fe Institute. Complexity. They discuss this and much much more in season 2. Although season 1 as well is super interesting.https://www.santafe.edu/culture/podcasts
       
 (DIR) Post #AntYhiyKKmH7e1FWyW by Erklaerbaer@mastodon.social
       2024-11-10T09:13:37Z
       
       0 likes, 0 repeats
       
       @futurebird @garymarcus the current episode is exactly about this, babies learning and AI learning
       
 (DIR) Post #AntZThKpivNQT2eUwy by Zeugs@social.cologne
       2024-11-10T09:22:12Z
       
       0 likes, 0 repeats
       
       @futurebird @Erklaerbaer @garymarcus It's more time to ask why should they? It's impressive on first sight what LLMs can do. Lots of people struggle to learn from text. If you compare it to a child and talking there is lots of other input data than the language. Also its not text but vowels created by special organs, perceived by special organs. Interpreted by evolutionary optimized hardware(brain). The problem is conceptual. The ignorant abstraction of humans by natural and computer science.
       
 (DIR) Post #AntZqt5ICpNdZGRON6 by TheDailyBurble@mastodon.social
       2024-11-10T09:26:28Z
       
       0 likes, 0 repeats
       
       @futurebird @Erklaerbaer @garymarcus A child sparks ideas in their own head about what is being said, concepts. AI just matches stuff up.
       
 (DIR) Post #AntagItU0SBgNpQyZM by futurebird@sauropods.win
       2024-11-10T09:35:48Z
       
       0 likes, 1 repeats
       
       @TheDailyBurble @Erklaerbaer @garymarcus That's part of it, but there are things about the ways that humans (and corvids!) process sound and words that I think are more basal and we could learn from that and make better models?LLM don't really have anything but statistics. It's just raw probability and reliance on the repetitiveness of language. If words cluster in one way most of the time, doing so again will seem like it's meaningful communication. What if this system was less lazy?
       
 (DIR) Post #Antb61r9yR3B4Vc8tE by TruthSandwich@fedi.truth-sandwich.com
       2024-11-10T09:40:24Z
       
       0 likes, 0 repeats
       
       @futurebird @Erklaerbaer @garymarcus According to people like Pinker (who's only worth listening to in his own field, which this is), children don't learn language, they learn specific languages.In other words, we don't start from a (groan) blank slate for languages. Computers do.
       
 (DIR) Post #AntcRZ8FYCzYJGRQ3M by TheDailyBurble@mastodon.social
       2024-11-10T09:55:32Z
       
       0 likes, 0 repeats
       
       @futurebird @Erklaerbaer @garymarcus I'm not sure. I remember learning language by disparate brain flashes and images which had nothing to do with language at all as far as I can remember that far back.Not association either, more something like dozing and odd connections. I have no idea how those eventually became conceptual, but for sure it wasn't mechanical "this word means this" stuff.
       
 (DIR) Post #AntfkShE0FrviJVDge by futurebird@sauropods.win
       2024-11-10T10:32:35Z
       
       0 likes, 0 repeats
       
       @miguelpergamon @TheDailyBurble @Erklaerbaer @garymarcus Please don't give them ideas.
       
 (DIR) Post #Antw2LAsJ2Xareuzcu by rafa_font@mastodon.online
       2024-11-10T13:35:01Z
       
       0 likes, 0 repeats
       
       @futurebird @Erklaerbaer @garymarcus I'm here to say that my children do consume vast amounts of energy. Especially MY energy 😇