Post B91MoWeBR1kTLeJSXA by ariadne@social.treehouse.systems
 (DIR) More posts by ariadne@social.treehouse.systems
 (DIR) Post #B918W2KI1uNQjsIUt6 by ariadne@social.treehouse.systems
       1 likes, 2 repeats
       
       @shituationist lots of people have asked me if I think LLMs will result in any of the promises the LLM companies have made, all the way up to and including self-modifying models (superintelligence), and I keep saying "no"and these people who ask me this, who are typically pretty smart people, but none of them have any actual expertise in ML other than memeing around with ollama or vLLM or maybe tensorflow/transformers for a sophisticated experimenter, so they choose the side of confirmation bias and tell me I'm definitely wrong on this 🙃if you look closely, you'll notice that OpenAI is already hitting scalability problems keeping ChatGPT up and running, and we know they are having scaling problems because of how the outages are shaped: they come as micro-outages, where inference requests will sporadically fail for short periods of time.  this type of availability problem is basically almost always caused by lack of horizontal scaling. and that's because they simply don't have the inference capacity.  they buy all the GPUs, all the RAM, its still not enough.and the bill is coming due at the end of this quarter for OpenAI, though apparently nvidia have proposed refinancing the debt, so who knows?
       
 (DIR) Post #B91GU0jNmzskqTZyxE by Su_G@aus.social
       0 likes, 0 repeats
       
       @ariadne So interesting re “scalability problems” -  dovetails with the anecdata of friends using ChatGPT (yes I know, but I know lots of different people & some of them use GenAI &/or LLMs & for lots of different things, but this is what those using ChatGPT report). #ScalabilityProblems #scalabilityIssues #ChatGPT #Anecdata
       
 (DIR) Post #B91LR2II1N4IaxJ8aG by claus@fosstodon.org
       0 likes, 0 repeats
       
       @ariadne @shituationist What does scaling/inference problems due to high demand have to do with the capabilities of those systems in the long run? It's like saying cars will never go 100km/h because there are no streets for that.
       
 (DIR) Post #B91LfBIdKrRQZzDINU by ariadne@social.treehouse.systems
       0 likes, 1 repeats
       
       @shituationist @claus I mean the answer, bluntly, is obvious.with current tech: more capabilities = more sub-experts = more parametersmore parameters = more parameters per inference run = more compute necessary to support the inferenceand training of course is much more intensive
       
 (DIR) Post #B91MKJbFNdUoC9XAQ4 by claus@fosstodon.org
       0 likes, 0 repeats
       
       @ariadne @shituationist Yeah, I see what you mean. But doesn't that mean that it will just be very expensive so demand is reduced? I don't see where this introduces a limit of how capable a model can get.
       
 (DIR) Post #B91MoWeBR1kTLeJSXA by ariadne@social.treehouse.systems
       0 likes, 1 repeats
       
       @shituationist @claus on its own, it doesn't.but when you include the pressure to have a product and have users and have to maintain quality of service for those users (which they are failing to do, everytime you have to retry a query they have failed), then you have to divert more resources to consumer inference, and away from trainingthis is just the reality of hitting a capacity wall if you are an AI lab pursuing LLM technology over more composable approaches
       
 (DIR) Post #B91NCdKMFXC115XEgq by claus@fosstodon.org
       0 likes, 0 repeats
       
       @ariadne @shituationist Get it. I'm curious how this will play out in the long run, though. Maybe compute will be so cheap in 10 years that it won't be a limiting factor, (but others).
       
 (DIR) Post #B91O54pEEGFhwAw2sK by ariadne@social.treehouse.systems
       0 likes, 1 repeats
       
       @shituationist @claus LLMs won't be around in 10 years unless the capacity wall the labs face today gets solved.
       
 (DIR) Post #B91ORypKhG4hzhuBBQ by miniBill@mastodon.uno
       0 likes, 0 repeats
       
       @ariadne @shituationist @claus I mean, open weight models are not going to spontaneously disappear (as much as I'd like to)
       
 (DIR) Post #B91OvVGZARDH06ilhQ by ariadne@social.treehouse.systems
       0 likes, 1 repeats
       
       @shituationist @claus @miniBill my point has nothing to do with that.  if the frontier LLMs cannot scale further, then it creates opportunity for competing technology which uses the capacity more efficiently.and that will trickle into the open weight models too.
       
 (DIR) Post #B91P96WJy7iAJ5ZknI by miniBill@mastodon.uno
       0 likes, 0 repeats
       
       @ariadne @shituationist @claus oh 💭 yeah, I can see that
       
 (DIR) Post #B91QV3rCeQ8QN8NQTg by ariadne@social.treehouse.systems
       0 likes, 0 repeats
       
       @miss_rodent @shituationist @claus yeah, there's basically no path for them to get the capacity they need.  it would be miraculous.
       
 (DIR) Post #B92H6O7BUVoBX0yK3s by mcc@mastodon.social
       0 likes, 0 repeats
       
       @ariadne For awhile I imagined that the nature of all these loans was Microsoft buying OpenAI in a weird way and over a long slow period of time. Like the loans aren't meant to be paid back, they're meant to go bad in a way that ends with Microsoft owning the company. Except now it appears they've sold the company three times, once to Microsoft, once to NVidia, and once to Oracle? So I don't know who gets the assets in a bankruptcy proceeding.