[HN Gopher] Why AI is still dumb and not scary at all (pt. 1)
___________________________________________________________________
Why AI is still dumb and not scary at all (pt. 1)
Author : tejonutella
Score : 38 points
Date : 2025-03-08 21:25 UTC (1 hours ago)
(HTM) web link (tejo.substack.com)
(TXT) w3m dump (tejo.substack.com)
| magic_hamster wrote:
| This seems to just ignore what's going on in the world. Some
| parts of the software development domain are definitely being
| affected by AI. I see a particular shift in web development and
| possibly embedded systems firmware development (or any other
| field where there is an exact specification of what you're
| supposed to deliver).
|
| LLMs don't need to be AGI. There's a race to adjust testing,
| architecture and infrastructure to fit AI agents who could
| possibly change them as needed. Some tools are already pretty
| good and they will only get better.
|
| Long story short, we are definitely going to see massive shift in
| jobs in the coming years.
| bumby wrote:
| You make an interesting point about embedded specifications.
| But doesn't that assume two things: 1) the specs are clear and
| complete, 2) we're ok trusting AI to develop safety critical
| software in those subsets of embedded applications?
| bmitc wrote:
| AI is more scary because it is so dumb.
| esafak wrote:
| This is a shallow and dated article, even when it came out in
| 2023. LLMs are dumb because they can't multiply, and generalize?
| They can easily write programs and calculate anything you want,
| and machine learning is all about generalizing.
| Hot take: if your job can be partially or wholly eliminated by
| AI, that's a GOOD THING. If your job has patterns that
| predictable or labor that routine, AI automation is a GOOD THING.
|
| What if it lead to widespread under- or unemployment? This social
| upheaval would lead to people setting aside the niceties of
| civilization as they fight to meet their basic needs. And such
| people elect bad governments, which exacerbates the problem.
|
| One might also be scared in a physical sense. The race is afoot
| to develop physical robots on par with humans. Imagine that as
| your policeman, or soldier, bringing democracy to a country near
| you.
|
| https://www.yahoo.com/tech/china-tech-contest-features-robot...
| narcraft wrote:
| I'll tell them, but no one will listen!
| Salgat wrote:
| The entire point is that LLMs don't think, they're just
| regressions of information you'd find on the internet. And
| automation is a good thing. 90% of Americans were farmers in
| the 1700s; automation completely revolutionized that. Your
| final point is still a concern in general, specifically what
| will happen when menial labor no longer exists. At that point,
| you have to hope that all the wealth generated from automation
| will be redistributed back in the form of a basic income,
| otherwise yes, we'll have a lot of people with no means to
| survive.
| esafak wrote:
| They are not interpolating, which is what I think you meant
| to say, except for a loose definition humans would meet too.
|
| What do you think of the latest thinking models, and what is
| your test of thinking?
| r0ckarong wrote:
| The scariest thing about innovation is the inability of a human
| mob to distinguish between lateral movement and intent.
| dgfitz wrote:
| *why llm models will always be dumb
| ohgr wrote:
| Dumb stuff is what I want. Classifiers and denoise are really
| useful.
|
| What I don't want is a shitty chat bot that barely understands
| what you ask it and generates shitty code by sticking a corpus of
| shitty code together then leaving me with the unconstrained job
| of working out if it has any land mines in it. Then multiply that
| by 50 engineers. Then tell me that's the future. Because if it is
| we are totally fucked. Our decline will be noted by the general
| decline in capacity to do anything.
| Garlef wrote:
| kids these days, right?
| chrismcb wrote:
| Because it isn't AI.
| ed-209 wrote:
| A gun is not smart or magical but is nevertheless a powerful tool
| that can be scary depending on who is holding it. Accordingly, I
| worry less about my occupation and more about the moral character
| of those wielding it. Further i worry about "smart" people who
| have not been acquainted with the dark side of human nature
| facilitating bad actors.
| ohgr wrote:
| Statistically speaking though if you leave guns lying around
| everywhere or introduce guns as a service for only $10/month
| and tell everyone that guns will solve all your problems, then
| you're going to end up with fuckwits with guns.
| vivekd wrote:
| [delayed]
| wat10000 wrote:
| A gun's purpose isn't to be smart. The gun equivalent of this
| post would be "Why guns still aren't very good at killing" and
| that would be a serious problem for guns if that were true.
| dgfitz wrote:
| AI doesn't exist. large language models are what they are,
| language models.
| haswell wrote:
| This is a frustrating piece on many levels, but mostly because it
| doesn't really scratch the surface of what is worrisome about AI.
| It sets up straw men and knocks them down, but not much else.
|
| It seems to boil down to:
|
| 1. LLMs aren't actually _" learning"_ the way humans do so we
| shouldn't be worried
|
| 2. LLMs don't actually _" understand"_ anything so we shouldn't
| be worried
|
| 3. Technology has always been advancing and we've always been
| freaking out about that, so we shouldn't be worried
|
| 4. If your job is automate-able, it probably _should_ be
| eliminated anyway
|
| What's scary is not that these models are smarter than us, but
| that we are dumb enough to deploy them in critical contexts and
| trust the output they generate.
|
| What's scary isn't that these models are so good they'll replace
| us, but that despite how limited they are, someone will make the
| decision to replace humans anyway.
|
| What's scary isn't that LLMs will displace good developers, but
| that LLMs put the power of development in the hands of people who
| have no idea what they're wielding.
|
| > _Sure, with a millions upon millions of training examples, of
| course you can mimic intelligence. If you already know what's
| going to be on the test, common patterns for answers in the test,
| or even the answer key itself, then are you really intelligent?
| OR are you just regurgitating information from billions of past
| tests?_
|
| How different are humans from this description in actuality? What
| are we if not the results of a process that has been optimized by
| millions upon millions of iterations over long periods of time?
| war-is-peace wrote:
| > What's scary isn't that these models are so good they'll
| replace us, but that despite how limited they are, someone will
| make the decision to replace humans anyway.
|
| is this a real threat? if a system/company decides to replace a
| human with something that is less capable wouldn't that just
| result in it becoming irrelevant/bankrupt as it is replaced by
| other companies doing the same thing the more efficient (and in
| this case traditional) way?
| ed-209 wrote:
| Not necessarily. Imagine a health insurance provider even
| partially automating their claim (dis)approval process - it
| could be both lucrative and devastating.
| haswell wrote:
| Adding to this, government use cases would be most likely
| to cause issues because they're often relevant regardless
| of how badly they suck.
|
| There are already active discussions about AI being used in
| government for "efficiency" reasons.
| war-is-peace wrote:
| i suppose that links back to the other comment i made -
| is hype the root issue you are trying to get at?
|
| would be interesting to see what examples there are of
| this in recent history
| war-is-peace wrote:
| worse for the consumer or the provider? if the llm is going
| to fundamentally do a "worse" job no matter what the
| incentive (whatever that is, maximising profit, maximising
| claims, whatever that may be) we will end up with the "more
| efficient" system in charge
|
| the counterpoint to this (which i guess is the tenet of the
| original comment?) is that hype can shadow good judgement
| for a short period of time?
| philjohn wrote:
| Already happened. With predictable results.
|
| https://www.youtube.com/shorts/Y7QPXzDmloI?embeds_referring
| _...
| IanCal wrote:
| Capability and efficiency are very different things.
|
| Consumers will absolutely buy worse buy cheaper
| products/services.
| jdietrich wrote:
| Has any company ever truly believed that outsourcing their
| customer service to India would improve the experience for
| their customers? No, but they did it anyway, because it cut
| costs. AI can be obviously worse than humans and still put
| them out of work, because GPUs are much cheaper than people.
| borisk wrote:
| "Change is inevitable. You can't stop the railroad as they used
| to say. It's going to kill some jobs but not all of them."
|
| I personally don't worry about jobs. AI is progressing very fast
| (there are a lof of smart folks working on it, there are a ton of
| money invested and there's a lot of demand from businesses,
| governments and individuals). Human inteligence stays the same. I
| think it's likely that sometime soon AI models will become more
| inteligent than an average human. And then more inteligent than
| the smartest human. And then more inteligent than the whole human
| race combined.
|
| Let's say some worms 600 million years ago could think. And they
| consider should they kill all mutants and stay forever the
| pinnacle of evolution as they are or allow some of them to evolve
| into fish, and then mammals and eventually inteligent humans. I
| think we are in a position like this - we are currently the
| pinacle of "creation" in the known universe - do we want to stay
| this way - by blocking AGI progress, or do we want to allow minds
| far greater than us to evolve from current LLMs at the cost of
| probable human extinction eventually.
| Herring wrote:
| I'm pretty sure you couldn't trust me to multiply 20-digit
| numbers either.
| asadotzler wrote:
| But you could trust a $3 drug-store checkout isle slim wallet
| calculator (with solar panels for recharging) to perform those
| calculations and you'd get more nines of reliability from that
| calculator than the LLM. And it didn't cost hundreds of
| billions of dollars to develop the calculator, so you won't end
| up paying thousands of dollars for it.
|
| That LLMs cannot beat a $3 calculator is because they're not
| fit for purpose. Everything they offer is a hallucination. Just
| because that fabrication matches with reality some of the time
| does not make it good. Reliability matters and these things
| just don't have it.
| jdietrich wrote:
| If you think an LLM is unfit for purpose, try getting a
| calculator to write poetry.
| ttul wrote:
| Ugh, I really dislike this sort of hand-waving blog post. The
| author cobbles together an argument dispelling all concerns about
| AI proliferation by assembling a few Tweets. Don't mistake this
| for scholarship. I believe the true answer to the question of
| whether we should be frightened of these gigantic models is far
| more nuanced and way less certain.
|
| A more honest blog post would read something like this: Big Ai
| models are interesting and useful. And there's the potential for
| danger. We should be cautious and make sure we have some bright
| scholars studying what might happen if various scenarios play
| out. Just as we would want to be in response to any similarly
| massive advancement in technology.
| robwwilliams wrote:
| We have two words we need to define: "sentience" and "thinking".
|
| Sentience can be defined as a basic function of most living
| systems (viruses excluded). You may disagree but to me this word
| connotes "actively responsive to perturbations". This is part of
| the auopoietic definition of life introduced by Maturana and
| Valera in the 1970s and 1980s. If you buy this definition then
| "sentience" is a low bar and not what we mean by AI, let alone by
| AGI.
|
| The word "thinking" implies more than just consciousness. Many of
| us accept that cats, dogs, pigs, chimps, crows, parrots, and
| whales are conscious. Perhaps not an ant or a spider, but yes,
| probably rats since rats play and smile and clearly plan. But few
| of us would be likely to claim that a rat thinks in a way that we
| think. Sure, a rat analyzes its environment and comes to
| decisions about best actions, but that does not require what many
| of us mean by thinking.
|
| We usually think of thinking as a process of
| reflection/recursion/self-evaluation of alternative scenarios.
| The key word here is recursion. and that gets us to "self-
| consciousness"--a word that embeds recursion.
|
| I operationally define thinking as a process that involves
| recursive review and reflection/evaluation of alternatives with
| the final selection of an "action"--either a physical action or
| an action that is a "thought conclusion".
|
| What AI/LMM transformers do not have is deep enough recursion or
| access to memory of preceding states. They cannot remember and
| they do not have a meta-supervisor that lets them modulate their
| own attentional state. So I would agree that they are still
| incapable of what most if us call thinking.
|
| But how hard is adding memory access to any number of current
| generation AI systems?
|
| How hard us it to add recursion and chain-of-thought?
|
| How hard will it be to enable an AI system to attend to its own
| attention?
|
| In my optimistic view these are all close to solved problems.
|
| In my opinion the only thing separating us from true AGI are a
| few crucial innovations in AI architecture---mainly self-control
| of attention---and under 100k lines of code.
|
| Please argue! (I've had many arguments with Claude on this
| topic).
| lostmsu wrote:
| I totally buy the consciousness bit. I mean why did people even
| need to dive deep into "philosophical" discussions when there's
| nothing wrong with "a conscious horse" vs "an unconscious
| horse" in the first place which is a dead simple distinction?
| Sometimes an unconscious horse is just a horse without
| consciousness.
| imiric wrote:
| The distinction from human thinking and what the current AI
| hype cycle calls "thinking" is that all the machine model is
| doing is outputting the most probable text patterns based on
| its training data. We can keep adding layers on top of this,
| but that's all it ultimately is. The machine has no real
| understanding of what the text it's outputting means or what
| that text represents in the real world. It can't have an
| intuitive grasp of these concepts in the same way that a human
| with our 5 senses can gather over our lifetime.
|
| This is why it's disingenuous to anthropomorphize any process
| of the current iteration of machine learning. There's no
| thinking or reasoning involved. None.
|
| This isn't to say that this technology can't be very useful.
| But let's not delude ourselves thinking that we're anywhere
| close to achieving AGI.
|
| "Arguing" about this topic with an LLM is pointless.
| seydor wrote:
| Crap article with zero insight that wasn't worth my time.
|
| There are better ways to argue the same position but it's
| probably indefensible.
|
| Neural networks are fundamentally approximators, both when they
| are approximating long-range relations between concepts (like
| LLMs do) or when they denoise some noise for images. They
| approximate thought and intelligence, because we have coded it in
| our writings. Our writings are a complete fingerprint of the
| thought process, because we are able to pick up any thought
| process from a book, without ever seeing or otherwise having
| contact with the author. Therefore there is an increasingly more
| high-fidelity "real thought" in there.
|
| > that in and of itself is a monumental achievement but there is
| no real thought involved.
|
| Pretty much anything that requires our current level of thought
| is therefore reachable with ANNs now that we know how to train
| them in depth and width.
|
| The real question is whether this is enough. We want ASI, but our
| texts only have AGI, and everything that comes from biology
| (including intelligence) is logarithmic in scale. There is zero
| evidence that language models will ever learn to create abstract
| entities better than any collection of humans do. AI companies
| are advertising armies of PhD students, but we already have
| millions of PhD students , yet our most pressing problems have
| not made a lot of progress for decades. That's what should be
| worrying to us, not the fact that we will all lose our jobs.
| metalman wrote:
| there is scary, as in physicaly dangerous and then there is
| scarry, as in I cant stand looking at media any more, scary and
| scary as in my job is threatened or the price fixing scams have
| unhomed you or the app that provided my job leads, has been super
| optimised, and is no longer viable and scary as , all of the
| above can be ignored , defended, as the app "just did it defence"
| with no one liable for the abuses, scary
___________________________________________________________________
(page generated 2025-03-08 23:00 UTC)