[HN Gopher] What happens when people don't understand how AI works
___________________________________________________________________
What happens when people don't understand how AI works
Author : rmason
Score : 245 points
Date : 2025-06-08 20:25 UTC (1 days ago)
(HTM) web link (www.theatlantic.com)
(TXT) w3m dump (www.theatlantic.com)
| jdkee wrote:
| Someone said, "The AI you use today is the worst AI that you will
| ever use."
| threeseed wrote:
| I can't tell any difference between Claude 3.5, 3.7 and 4 for
| coding.
|
| So today is the same AI I used last year. And based on current
| trajectory same I will use next year.
| sroussey wrote:
| I can tell the difference between those versions of Claude
| quite easily. Not 10x better each version, but each is more
| capable and the errors are fewer.
| assimpleaspossi wrote:
| But errors nonetheless.
| plemer wrote:
| You won't find perfection anywhere
| th0ma5 wrote:
| The imperfections of people are less imperfect than the
| machines.
| 8note wrote:
| errors are all over the place no matter what. the
| question is how predictable they are, and if they can be
| spotted as they show up.
|
| the best bugs are the ones that arent found for 5 years
| smcleod wrote:
| There is certainly a difference, however Anthropic did really
| well with 3.5 - far, far better than any other provider could
| do, so the steps from there been more incremental while other
| providers have been playing catch up (for example Google's
| Gemini 2.5 Pro is really their first model that's actually
| useful for coding in any way).
| user568439 wrote:
| Someone also said the current AI is the less biased.
|
| Future AI's will be more powerful but probably influenced to
| push users to spend money or have a political opinion. So they
| may enshitify...
| Pingk wrote:
| Given that no models are profitable for the parent company
| afaik, it's only a matter of time before the money-squeezing
| begins
| stirfish wrote:
| <think>I will reply with an example of bias in large language
| models. This comment seems unrelated to farmers in South
| Africa. </think>
|
| Ultimately these machines work for the people who paid for
| them.
| add-sub-mul-div wrote:
| That doesn't take into account how it will be used against its
| customers once they're dependent on it. As any other service
| inevitably will be.
|
| It's like if we'd said the Youtube we used in 2015 was going to
| be the worst Youtube we'd ever use.
| andy99 wrote:
| Is the internet today better than it's ever been? I can easily
| imagine LLMs becoming ad-riden, propagandized, cloudlfare
| intermediated, enshittified crap the way most of the internet
| has.
| davidcbc wrote:
| Someone could have said "The Google you use today is the worst
| Google you will ever use" 15 years ago and it may have sounded
| wise at the time.
| dwaltrip wrote:
| That's a good point. But on the other hand Google search
| usefulness didn't have "scaling laws" / lines on capability
| graphs going up and up...
| moduspol wrote:
| Maybe not 15 years ago, but they did early on. And then at
| some point, the lines kind of leveled off.
| rhubarbtree wrote:
| If there is one cliche that I'd like to outlaw, it is this one.
| It's a vacuous tautology and yet I hear so many people parrot
| it all the time.
|
| There are many reasons to believe LLMs in particular are not
| going anywhere fast.
|
| We need major breakthroughs now, and "chain of thought" is not
| one.
| layer8 wrote:
| https://archive.is/NJ9K0
| mettamage wrote:
| > Few phenomena demonstrate the perils that can accompany AI
| illiteracy as well as "Chatgpt induced psychosis," the subject of
| a recent Rolling Stone article about the growing number of people
| who think their LLM is a sapient spiritual guide. Some users have
| come to believe that the chatbot they're interacting with is a
| god--"ChatGPT Jesus," as a man whose wife fell prey to LLM-
| inspired delusions put it--while others are convinced, with the
| encouragement of their AI, that they themselves are metaphysical
| sages in touch with the deep structure of life and the cosmos. A
| teacher quoted anonymously in the article said that ChatGPT began
| calling her partner "spiral starchild" and "river walker" in
| interactions that moved him to tears. "He started telling me he
| made his AI self-aware," she said, "and that it was teaching him
| how to talk to God, or sometimes that the bot was God--and then
| that he himself was God."
|
| This sounds insane to me. When we are talking about safe AI use,
| I wonder if things like this are talked about.
|
| The more technological advancement goes on, the smarter we need
| to be in order to use it - it seems.
| polotics wrote:
| Psychosis will find anything as a seed idea to express itself,
| even as basic a pattern as someone walking in lockstep with the
| soon-to-be patient can trigger a break. So it's not surprising
| that LLM chats would do the same.
| daveguy wrote:
| Yeah, but should we have psychosis force multipliers?
| Animats wrote:
| We have that now, in social media. If you create some way
| for large numbers of people with the same nutty beliefs to
| easily communicate, you get a psychosis force multiplier.
| Before social media, nuttyness tended to be diluted by the
| general population.
| daveguy wrote:
| Completely agree with you about social media. I'm not a
| fan of social media algorithms and believe they also
| produce much more harm than benefit.
| hackingonempty wrote:
| I think insane and lonely people are definitely on the safety
| radar.
|
| Even if todays general purpose models and models made by
| predators can have negative effects on vulnerable people, LLMs
| could become the technology that brings psych care to the
| masses.
| Animats wrote:
| > Few phenomena demonstrate the perils that can accompany AI
| illiteracy as well as "Chatgpt induced psychosis," the subject
| of a recent Rolling Stone article about the growing number of
| people who think their LLM is a sapient spiritual guide.
|
| People have been caught in that trap ever since the invention
| of religion. This is not a new problem.
| staunton wrote:
| It doesn't have to be new to get worse and be important as a
| consequence of this stuff...
| nkrisc wrote:
| Crazy people will always find a way to be crazy. It's not
| surprising that many of these cases have a religious nature to
| them.
| stirfish wrote:
| I'll admit, the first time I started ollama locally, I asked it
| if I would hurt it if I turned it off. I have a master's degree
| in machine learning and I know it's silly, but I still did it.
| carlosjobim wrote:
| We were warned:
|
| "You shall not make idols for yourselves or erect an image or
| pillar, and you shall not set up a figured stone in your land
| to bow down to it, for I am the LORD your God."
|
| A computer chip is a stone (silicon) which has been engraved.
| It's a graven image.
|
| Anything man-made is always unworthy of worship. That includes
| computer programs such as AI. That includes man-made ideas such
| as "the government", a political party, or other abstract
| ideas. That also includes any man or woman. But the human
| natural instinct is to worship a king, pharaoh or an emperor -
| or to worship a physical object.
| streb-lo wrote:
| I mean, the bible is a man made pile of crap too...
| carlosjobim wrote:
| If God or The Gods are defined as not being man-made, then
| each person will be able to find their own interpretation
| and understanding. As contrary to man-made objects and
| concepts. Most modern people worship "the government" or
| "the state", even though there is no dispute whether it was
| created by man or not and whether it acts under the
| influence of man or not.
| BirAdam wrote:
| Whether or not you are religious, channeling the human
| impulse to worship into something singular, immaterial,
| eternal, without form, and with very precise rules to not
| murder, lie, covet, etc... is quite useful for human
| organization.
| kelseyfrog wrote:
| LLMs are divinatory instruments, our era's oracle, minus the
| incense and theatrics. If we were honest, we'd admit that
| "artificial intelligence" is just a modern gloss on a very old
| instinct: to consult a higher-order text generator and search for
| wisdom in the obscure.
|
| They tick all the boxes: oblique meaning, a semiotic field, the
| illusion of hidden knowledge, and a ritual interface. The only
| reason we don't call it divination is that it's skinned in dark
| mode UX instead of stars and moons.
|
| Barthes reminds us that all meaning is in the eye of the reader;
| words have no essence, only interpretation. When we forget that,
| we get nonsense like "the chatbot told him he was the messiah,"
| as though language could be blamed for the projection.
|
| What we're seeing isn't new, just unfamiliar. We used to read
| bones and cards. Now we read tokens. They look like language, so
| we treat them like arguments. But they're just as oracular,
| complex, probabilistic signals we transmute into insight.
|
| We've unleashed a new form of divination on a culture that
| doesn't know it's practicing one. That's why everything feels
| uncanny. And it's only going to get stranger, until we learn to
| name the thing we're actually doing. Which is a shame, because
| once we name it, once we see it for what it is, it won't be half
| as fun.
| Arainach wrote:
| >When we forget that, we get nonsense like "the chatbot told
| him he was the messiah," as though language could be blamed for
| the projection.
|
| Words have power, and those that create words - or create
| machines that create words - have responsibility and liability.
|
| It is not enough to say "the reader is responsible for meaning
| and their actions". When people or planet-burning random matrix
| multipliers say things and influence the thoughts and behaviors
| of others there is blame and there should be liability.
|
| Those who spread lies that caused people to storm the Capitol
| on January 6th believing an election to be stolen are
| absolutely partially responsible even if they themselves did
| not go to DC on that day. Those who train machines that spit
| out lies which have driven people to racism and genocide in the
| past are responsible for the consequences.
| kelseyfrog wrote:
| "Words have no essential meaning" and "speech carries
| responsibility" aren't contradictory. They're two ends of the
| same bridge. Meaning is always projected by the reader, but
| that doesn't exempt the speaker from shaping the terrain of
| projection.
|
| Acknowledging the interpretive nature of language doesn't
| absolve us from the consequences of what we say. It just
| means that communication is always a gamble: we load the dice
| with intention and hope they land amid the chaos of another
| mind.
|
| This applies whether the text comes from a person or a model.
| The key difference is that humans write with a theory of
| mind. They guess what might land, what might be misread, what
| might resonate. LLMs don't guess; they sample. But the
| meaning still arrives the same way: through the reader,
| reconstructing significance from dead words.
|
| So no, pointing out that people read meaning into LLM outputs
| doesn't let humans off the hook for their own words. It just
| reminds us that all language is a collaborative illusion,
| intent on one end, interpretation on the other, and a vast
| gap where only words exist in between.
| krisoft wrote:
| This sounds very wise but doesn't seem to describe any of my
| use cases. Maybe some use cases are divination but it is a
| stretch to call all of them that.
|
| Just looking at my recent AI prompts:
|
| I was looking for the name of the small fibers which form a
| bird's feather. ChatGPT told me they are called "barbs". Then
| using straight forward google search i could verify that indeed
| that is the name of the thing i was looking for. How is this
| "divination"?
|
| I was looking for what is the g-code equivalent for galvo fiber
| lasers are and ChatGPT told me there isn't really one. The
| closest might be the sdk of ezcad, but it also listed several
| other opensource control solutions too.
|
| Wanted to know what are the hallmarking rules in the UK for an
| item which consist of multiple pieces of sterling silver held
| together by a non-metalic part. (Turns out the total weight of
| the silver matters, while the weight of the non-metalic part
| does not count.)
|
| Wanted to translate the hungarian phrase "besurrano tolvaj"
| into english. Out of the many possible translations chatGPT
| provided "opportunistic burglar" fit the best for what I was
| looking for.
|
| Wanted to write an sql alchemy model and i had an approximate
| idea of what fields i needed but couldn't be arsed to come up
| with good names for them and find the syntax to describe their
| types. ChatGPT wrote it for me in seconds what would have taken
| me at least ten minutes otherwise.
|
| These are "divination" only in a very galaxy brained "oh man,
| when you open your mind you see everything is divination
| really". I would call most of these "information retrieval".
| The information is out there the LLM just helps me find it with
| a convenient interface. While the last one is "coding".
| kelseyfrog wrote:
| Sure, some people stepped up to the Oracle and asked how to
| conquer Persia. Others probably asked where they left their
| sandals. The quality of the question doesn't change the
| structure of the act.
|
| You presented clear, factual queries. Great. But even there,
| all the components are still in play: you asked a question
| into a black box, received a symbolic-seeming response,
| evaluated its truth post hoc, and interpreted its relevance.
| That's divination in structural terms. The fact that you're
| asking about barbs on feathers instead of the fate of empires
| doesn't negate the ritual, you're just a more practical
| querent.
|
| Calling it "information retrieval" is fine, but it's worth
| noticing that this particular interface feels like more than
| that, like there's an illusion (or a projection) of latent
| knowledge being revealed. That interpretive dance between
| human and oracle is the core of divination, no matter how
| mundane the interaction.
|
| I don't believe this paints with an overly broad brush. It's
| a real type of interaction and the subtle distinction focuses
| on the core relationship between human and oracle: seeking
| and interpreting.
| gaganyaan wrote:
| > you asked a question into a black box, received a
| symbolic-seeming response, evaluated its truth post hoc,
| and interpreted its relevance
|
| So any and all human communication is divination in your
| book?
|
| I think your point is pretty silly. You're falling into a
| common trap of starting with the premise "I don't like AI",
| and then working backwards from that to pontification.
| kelseyfrog wrote:
| Hacker News deserves a stronger counterargument than
| "this is silly."
|
| My original comment is making a structural point, not a
| mystical one. It's not saying that using AI feels like
| praying to a god, it's saying the interaction pattern
| mirrors forms of ritualized inquiry: question - symbolic
| output - interpretive response.
|
| You can disagree with the framing, but dismissing it as
| "I don't like AI so I'm going to pontificate" sidesteps
| the actual claim. There's a meaningful difference between
| saying "this tool gives me answers" and recognizing that
| the process by which we derive meaning from the output
| involves human projection and interpretation, just like
| divination historically did.
|
| This kind of analogy isn't an attack on AI. It's an
| attempt to understand the human-AI relationship in
| cultural terms. That's worth engaging with, even if you
| think the metaphor fails.
| hexaga wrote:
| > Hacker News deserves a stronger counterargument than
| "this is silly."
|
| Their counterargument is that said structural definition
| is overly broad, to the point of including any and all
| forms of symbolic communication (which is all of them).
| Because of that, your argument based on it doesn't really
| say anything at all about AI or divination, yet still
| seems 'deep' and mystical and wise. But this is a seeming
| only. And for that reason, it is silly.
|
| By painting all things with the same brush, you lose the
| ability to distinguish between anything. Calling all
| communication divination (through your structural
| metaphor), and then using cached intuitions about 'the
| thing which used to be called divination; when it was a
| limited subset of the whole' is silly. You're not talking
| about that which used to be called divination, because
| you redefined divination to include all symbolic
| communication.
|
| Thus your argument leaks intuitions (how that-which-was-
| divination generally behaves) that do not necessarily
| apply through a side channel (the redefined word). This
| is silly.
|
| That is to say, if you want to talk about the
| interpretative nature of interaction with AI, that is
| fairly straightforward to show and I don't think anyone
| would fight you on it, but divination brings baggage with
| it that you haven't shown to be the case for AI. In point
| of fact, there are many ways in which AI is not at all
| like divination. The structural approach broadens too far
| too fast with not enough re-examination of priors,
| becoming so broad that it encompasses any kind of
| communication at all.
|
| With all of that said, there seems to be a strong bent in
| your rhetoric towards calling it divination _anyway_ ,
| which suggests reasoning from that conclusion, and that
| the structural approach is but a blunt instrument to
| force AI into a divination shaped hole, to make 'poignant
| and wise' commentary on it.
|
| > "I don't like AI so I'm going to pontificate" sidesteps
| the actual claim
|
| What claim? As per ^, maximally broad definition says
| nothing about AI that is not also about everything, and
| only seems to be a claim because it inherits intuitions
| from a redefined term.
|
| > difference between saying "this tool gives me answers"
| and recognizing that the process by which we derive
| meaning from the output involves human projection and
| interpretation, just like divination historically did
|
| Sure, and all communication requires interpretation. That
| doesn't make all communication divination. Divination
| implies the notion of interpretation of something that is
| seen to be causally disentangled from the subject. The
| layout of these bones reveals your destiny. The level of
| mercury in this thermometer reveals the temperature. The
| fair die is cast, and I will win big. The loaded die is
| cast, and I will win big. Spot the difference. It's not
| structural.
|
| That implication of essential incoherence is what you're
| saying without saying about AI, it is the 'cultural
| wisdom and poignancy' feedstock of your arguments,
| smuggled in via the vehicle of structural metaphor along
| oblique angles that should by rights not permit said
| implication. Yet people will of course be generally
| uncareful and wave those intuitions through - presuming
| they are wrapped in appropriately philosophical guise -
| which is why this line of reasoning inspires such
| confusion.
|
| In summary, I see a few ways to resolve your arguments
| coherently:
|
| 1. keep the structural metaphor, discard cached
| intuitions about what it means for something to be
| divination (w.r.t. divination being generally wrong/bad
| and the specifics of how and why). results in an argument
| of no claims or particular distinction about anything,
| really. this is what you get if you just follow the logic
| without cache invalidation errors.
|
| 2. discard the structural metaphor and thus disregard the
| cached intuitions as well. there is little engagement
| along human-AI cultural axis that isn't also human-human.
| AI use is interpretative but so is all communication.
| functionally the same as 1.
|
| 3. keep the structural metaphor and also demonstrate how
| AI are not reliably causally entwined with reality along
| boundaries obvious to humans (hard because they plainly
| and obviously are, as demonstrable empirically in myriad
| ways), at which point go off about how using AI is
| divination because at this point you could actually say
| that with confidence.
| dingnuts wrote:
| You're misunderstanding the point of structural analysis.
| Comparing AI to divination isn't about making everything
| equivalent, but about highlighting specific shared
| structures that reveal how humans interact with these
| systems. The fact that this comparison can be extended to
| other domains doesn't make it meaningless.
|
| The issue isn't "cached intuitions" about divination, but
| rather that you're reading the comparison too literally.
| It's not about importing every historical association,
| but about identifying specific parallels that shed light
| on user behavior and expectations.
|
| Your proposed "resolutions" are based on a false
| dichotomy between total equivalence and total abandonment
| of comparison. Structural analysis can be useful even if
| it's not a perfect fit. The comparison isn't about
| labeling AI as "divination" in the classical sense, but
| about understanding the interpretive practices involved
| in human-AI interaction.
|
| You're sidestepping the actual insight here, which is
| that humans tend to project meaning onto ambiguous
| outputs from systems they perceive as having special
| insight or authority. That's a meaningful observation,
| regardless of whether AI is "causally disentangled from
| reality" or not.
| gaganyaan wrote:
| > humans tend to project meaning onto ambiguous outputs
| from systems they perceive as having special insight or
| authority
|
| This applies just as well to other humans as it does AI.
| It's overly-broad to the point of meaninglessness.
|
| The insight doesn't illuminate.
| albedoa wrote:
| > That's a meaningful observation, regardless of whether
| AI is "causally disentangled from reality" or not.
|
| And regardless of how many words someone uses in their
| failed attempt at "gotcha" that nobody else is playing.
| There are certainly some folks acting silly here, and
| it's not the vast majority of us who have no problem
| interpreting and engaging with the structural analysis.
| nosianu wrote:
| > _So any and all human communication is divination in
| your book?_
|
| Words from an AI are just words.
|
| Words in a human brain have more or less (depending on
| the individual's experiences) "stuff" attached to them:
| From direct sensory inputs to complex networks of
| experiences and though. Human thought is mainly not based
| on words. Language is an add-on. _(People without
| language - never learned, or sometimes temporarily
| disabled due to drugs, or permanently due to injury,
| transient or permanent aphasia - are still consciously
| thinking people.)_
|
| Words in a human brain are an expression of deeper
| structure in the brain.
|
| Words from an AI have nothing behind them but word
| statistics, devoid of any real world, just words based on
| words.
|
| Random example sentence: "The company needs to expand
| into a new country's market."
|
| When an AI writes this, there is no real world meaning
| behind it whatsoever.
|
| When a fresh out of college person writes this it's based
| on some shallow real world experience, and lots of
| hearsay.
|
| When an experienced person actually having done such
| expansion in the past says it a huge network of their
| experience with people and impressions is behind it, a
| feeling for where the difficulties lie and what to expect
| IRL with a lot of real-world-experience based detail.
| When such a person expands on the original statement
| chances are highest that any follow-up statements will
| also represent real life quite well, because they are
| drawn not from text analysis, but from those deeper
| structures created by and during the process of the
| person actually performing and experiencing the task.
|
| But the words can be exactly the same. Words from a human
| _can_ be of the same (low) quality as that of an AI, if
| they just parrot something they read or heard somewhere,
| although even then the words will have more depth than
| the "zero" on AI words, because even the stupidest
| person has _some_ degree of actual real life forming
| their neural network, and not solely analysis of other 's
| texts.
| ben_w wrote:
| > People without language - never learned, or sometimes
| temporarily disabled due to drugs, or permanently due to
| injury, transient or permanent aphasia - are still
| consciously thinking people.
|
| There are 40 definitions of the word "consciousness".
|
| For the definitions pertaining to inner world, nobody can
| tell if _anyone_ besides themselves (regardless of if
| they speak or move) is conscious, and none of us can
| prove to anyone else the validity of our own claims to
| posess it.
|
| When I dream, am I conscious in that moment, or do I
| create a memory that my consciousness replays when I
| wake?
|
| > Words from an AI have nothing behind them but word
| statistics, devoid of any real world, just words based on
| words.
|
| > [...]
|
| > When a fresh out of college person writes this it's
| based on some shallow real world experience, and lots of
| hearsay.
|
| My required reading at school included "Dulce Et Decorum
| Est" by Wilfred Owen.
|
| The horrors of being gassed during trench warfare were
| alien to us in the peaceful south coast of the UK in
| 1999/2000.
|
| AI are limited, but what you're describing here is the
| "book learning" vs. "street smart" dichotomoy rather than
| their actual weaknesses.
| tempodox wrote:
| I can only agree with you. And I find it disturbing that
| every time someone points out what you just said, the
| counter argument is to reduce human experience and human
| consciousness to the shallowest possible interpretation
| so they can then say, "look, it's the same as what the
| machine does".
| kenjackson wrote:
| I think it's because the brain is simply a set of
| chemical and electrical interactions. I think some
| believe when we understand how the brain works it won't
| be some "soulful" other worldly explanation. It will be
| some science based explanation that will seem very
| unsatisfying to some that think of us as more than
| complex machines. The human brain is different than LLMs,
| but I think we will eventually say "hey we can make a
| machine very similar".
| tempodox wrote:
| It looks like you did exactly what I described in my
| parent comment, so it doesn't add anything of substance.
| Let's agree to disagree.
| lcnPylGDnU4H9OF wrote:
| The logic is that you preemptively shut down dissenting
| opinions so any comments with dissenting opinions are
| necessarily not adding anything of substance. They made
| good points and you simply don't want to discuss them;
| that does not mean the other commenter did not add
| substance and nuance to the discussion.
| rdtsc wrote:
| The deconstruction trick is a bit like whataboutism. It
| sort of works on a shallow level but it's a cheap shot.
| You can say "this is just a collections of bites and
| matrix multiplications". If it's humans -- "it's just
| simple neurons firing and hormones". Even if it's some
| object: "what's the big deal, it's just bunch of
| molecules and atoms".
| krisoft wrote:
| > some people stepped up to the Oracle and asked how to
| conquer Persia. Others probably asked where they left their
| sandals.
|
| And if the place would be any good at the second kind of
| queries you would call it Lost&Found and not the Oracle.
|
| > illusion (or a projection) of latent knowledge being
| revealed
|
| It is not an illusion. Knowledge is being revealed. The
| right knowledge for my question.
|
| > That interpretive dance between human and oracle is the
| core of divination, no matter how mundane the interaction.
|
| Ok, so if I went to a library, used a card index to find a
| book about bird feather anatomy, then read said book to
| find that the answer to my question is "barb" would you
| also call that "divination"?
|
| If i would have paid a software developer to turn my
| imprecise description of a database table into precise and
| thight code which can be executed would you also call that
| "divination"?
| anon-3988 wrote:
| The difference is between saying '"I want a hammer" and
| it magically pops in your hand' versus '"I want a hammer"
| and I have to chop some wood, gather some metals, heat it
| up...'.
|
| Both gets you a hammer, but I don't think anyone would
| call the latter magical/divine? I think its only
| "magical" simply because its incomprehensible...how does
| a hammer pops into reality? Of course, once we know
| EXACTLY how that works, then it ceases to be magical.
|
| Even if we take God, if we fully understand how He works,
| He would no longer be magical/divine. "Oh he created
| another universe? This is how that works..."
|
| The divinity comes from the fact that it is
| incomprehensible.
| tough wrote:
| I always like to compare tongue in cheek, llm's with
| I-ching
|
| https://en.wikipedia.org/wiki/I_Ching
| AlecSchueler wrote:
| Why?
| tough wrote:
| i copy pasted my comment and your question to chatgpt, so
| this isnt my answer but the AI's:
|
| make your own conclusions
|
| Because both LLMs and the I Ching function as mirrors for
| human interpretation, where: * The I Ching offers cryptic
| symbols and phrases--users project meaning onto them. *
| LLMs generate probabilistic text--users extract
| significance based on context.
|
| The parallel is:
|
| You don't get answers, you get patterns--and the meaning
| emerges from your interaction with the system.
|
| In both cases, the output is: * Context-sensitive * Open-
| ended * Interpreted more than dictated
|
| It's a cheeky way of highlighting that users bring the
| meaning, not the machine (or oracle).
| narrator wrote:
| You're describing what narrative archetype it is most
| similar to from ancient history, not what it actually is.
| zzbzq wrote:
| The LLMs do have "latent knowledge," indisputably, the
| latent knowledge is beyond reproach. Because what we do
| know about the "black box" is that inside it, is a database
| of not just facts, but understanding, and we know the model
| "understands" nearly every topic better than any human.
| Where the doubt-worthy part happens is the generative step,
| since it is tasked with producing a new "understanding"
| that didn't already exist, the mathematical domain of the
| generative function exceeds the domain of reality. And,
| second of all, because the reasoning faculties are far less
| proven than the understanding faculties, and many queries
| require reasoning about existing understandings to derive a
| good, new one.
| datadrivenangel wrote:
| LLMs have latent knowledge insofar as it can be distilled
| out of the internet...
| whilenot-dev wrote:
| *or any digitized proprietary works, just as long as they
| can be parsed correctly. don't worry, the means of how to
| optain these works doesn't seem to matter[0]
|
| [0]: https://www.arl.org/blog/training-generative-ai-
| models-on-co...
| falcor84 wrote:
| > Others probably asked where they left their sandals.
|
| This to me is massive. The Oracle of Delphi would have no
| idea where you left your sandals, but present day AIs
| increasingly do. This (emergent?) capability of combining
| information retrieval with flexible language is amazing,
| and its utility to me cannot be overstated, when I ask a
| vague question, and then I check the place where the AI led
| me to, and the sandals are indeed there.
|
| P.S. Thank you for introducing me to the word "querent"
| sdenton4 wrote:
| The particularly amazing part is that both the Oracle and
| the LLM said 'Right where you left them,' but only the
| LLM was correct.
| 725686 wrote:
| "how to conquer Persia" and "what is the name of the small
| fibers which form a bird's feather" are very different
| kinds of questions. There is no one right answer for the
| first. That is divination. The second is just information
| retrieval.
| diffeomorphism wrote:
| Which the LLM then does not do and instead makes up
| likely text.
|
| As prominent examples look at the news stories about
| lawyers citing nonexistent cases or publications.
|
| People think that LLMs do information retrieval, but they
| don't. That is what makes them harmful in education
| contexts.
| navane wrote:
| What about the times you didn't get a coherent answer and you
| gave up and looked elsewhere?
| JKCalhoun wrote:
| Almost proves it is not an oracle then, not perceived as
| one.
|
| Rephrasing: LLMs are the modern day oracle that we
| disregard when it appears to be hallucinating, embrace when
| it appears to be correct.
|
| The popularity of LLMs may not be that we see them as
| mystical, but rather that they're right more often than
| they're wrong.
|
| "That is not what I meant at all;
|
| That is not it, at all."
|
| -- T.S. Eliot
| dfxm12 wrote:
| _ChatGPT told me they are called "barbs". Then using straight
| forward google search i could verify that indeed that is the
| name of the thing i was looking for._
|
| Why not just start with a straight forward Google search?
| shagie wrote:
| If you are not familiar with the term, it can be hard to
| search for it.
|
| Google doesn't give you the answer (unless you're reading
| the AI summaries - then it's a question of which one you
| trust more). Instead it provides links to
| https://www.scienceofbirds.com/blog/the-parts-of-a-feather-
| and-how-feathers-work
| https://www.birdsoutsidemywindow.org/2010/07/02/anatomy-
| parts-of-a-feather/
| https://en.wikipedia.org/wiki/Feather
| https://www.researchgate.net/figure/Feather-structure-a-
| feather-shaft-rachis-and-the-feather-vane-barbs-and-
| barbules_fig3_303095497
|
| These then require an additional parsing of the text to see
| if it has what you are after. Arguably, one could read the
| Wiki article first and see if it has, but it's faster to
| ask ChatGPT and then verify - rather than search, scan, and
| parse.
| ramchip wrote:
| It gives you more effective search keywords. "Fibers in
| feathers" isn't too bad, but when it's quite vague like
| "that movie from the 70s where the guy drank whiskey and
| then there was a firefight and..." getting the name from
| the LLM makes it much faster to google.
| bandoti wrote:
| Funny I just entered "feather" into Merriam-Webster
| dictionary and there's your word "barb". Point being, people
| should use a dictionary/thesaurus before burning fuel on an
| AI.
|
| 1 a : any of the light, horny, epidermal outgrowths that form
| the external covering of the body of birds
|
| NOTE: Feathers include the smaller down feathers and the
| larger contour and flight feathers. Larger feathers consist
| of a shaft (rachis) bearing branches (barbs) which bear
| smaller branches (barbules). These smaller branches bear tiny
| hook-bearing processes (barbicels) which interlock with the
| barbules of an adjacent barb to link the barbs into a
| continuous stiff vane. Down feathers lack barbules, resulting
| in fluffy feathers which provide insulation below the contour
| feathers.
| dingnuts wrote:
| This is a great example because the LLM answer was
| insufficiently complete but if you didn't look up the
| result you wouldn't know. I think I remain an AI skeptic
| because I keep looking up the results and this kind of
| omission is more common than not.
| msgodel wrote:
| This stuff wasn't an issue because older societies had
| hierarchy which checked the mob.
|
| In a flat society every individual _must_ be able to perform
| philosophically the way aristocrats do.
| UncleOxidant wrote:
| We're seeing the effects of the flat society not being able
| to do this. Conspiracy theories, the return of mysticism
| (even things like astrology seem to be on the rise), distrust
| of experts, fear of the other, etc.
| mdp2021 wrote:
| > _we were honest_
|
| I am quite honest and the subset of users that fill your
| description - unconsciously treating text from deficient
| authors as tea leaves - have psychiatric issues.
|
| Surely many people consult LLMs because of the value within
| their right answers, which exist owing to having encoded
| information and some emergent idea processing, and attempting
| to tame the wrong ones. They consult LLMs because that's what
| we have, limited as it is, for some problems.
|
| Your argument falls immediately because people in the
| consultation of unreliable documents cannot be confused with
| people in the consultation of tools for other kinds of
| thinking: the thought under test is outside in the first case,
| inside in the second (contextually).
|
| You have fallen in a very bad use of 'we'.
| awongh wrote:
| > value within their right answers
|
| The thing is that LLMs provide plenty of answers where
| "right" is not a verifiable metric. Even in coding the idea
| of a "right" answer quickly gets fuzzy- _should I use CSS
| grid or flexbox here?_ _should these tables be normalized or
| not?_
|
| People simply have an unconscious bias towards the output
| just like they have an unconscious bias towards the same
| answer given by two real people they feel differently about-
| That is, the sort of thing all humans do (even if you swear
| that in all cases you are 100% impartial and logical).
|
| I think the impulse of ascribing intent and meaning to the
| output is there in almost all questions, it's just a matter
| of degrees (CSS question vs. meaning of life type question)
| mdp2021 wrote:
| > _LLMs provide plenty of answers where "right" is not a
| verifiable metric_
|
| I do not use them for that: I ask them for precise
| information. Incidentally, that one time in which I had to
| ask for a clever subtler explanation, it was possible to
| evaluate the quality of the answer - and I found myself
| pleasantly surprised (for once). What I said is, some
| people ask LLMs for information and explanation in absence
| of better and faster repositories - and it is just rational
| to do so. Those <<answers where "right" is not a verifiable
| metric>> are not relevant in this context. Some people use
| LLMs as <whatever>: yes, /some/ people. That some other
| people will ask LLMs fuzzy questions does not imply that
| they will accept them as oracles.
|
| > _bias ... all humans do_
|
| Which should, for the frame and amount in which the idea
| has some truth, have very little substantial weight and
| surely does not prove the "worshippers" situation depicted
| by the OP. You approach experience E in state S(t): that is
| very far from "wanting to trust" (which is just the twisted
| personality trait of some).
|
| > _the impulse of ascribing intent and meaning_ [...]
| _meaning of life_
|
| First, of all, no: there seems to be no <<intent and
| meaning>> in requests like "what is the main export of
| Kyrgyzstan", and people who ask an LLM about the meaning of
| life - as if dealing with an intelligent part instead of a
| lacking thing - pertain to a specific profile.
|
| If you have this addiction to dreaming, you are again
| requested to wake up. Yes, we know many people who
| stubbornly live in their own delirious world; they do not
| present themselves decently and they are quite distinct
| from people radicated in reality.
|
| I am reading that some people as if anthropomorphize LLMs,
| some daemonize LLMs - some people will even deify them -
| it's just stochastics. Guess what: some people reify some
| other people - and believe they are being objective. The
| full spectrum will be there. Sometimes justified, sometimes
| not.
| mdp2021 wrote:
| Addendum, because of real time events:
|
| I am reading in a www page (I won't even link it, because
| of decency):
|
| > _The authors[, from the psychology department,] also
| found that ... dialogues with AI-chatbots helped reduce
| belief in misinformation_ [...] _<<This is the first
| evidence that non-curated conversations with Generative AI
| can have positive effects on misinformation>>_
|
| Some people have their beliefs, but they can change them
| after discussing with LLMs (of all the ways). Some people
| are morons - we already knew that.
| fragmede wrote:
| We knew that, but that doesn't help it when the moron is
| the one holding the metaphorical (or even literal) gun.
| tim333 wrote:
| Maybe LLMs can be divinatory instruments but that sounds a bit
| highbrow going by my use.
|
| I use it more as a better Google search. Like the most recent
| thing I said to ChatGPT is "will clothianidin kill carpet
| beetles?" (turns out it does by the way.)
| suddenlybananas wrote:
| Until it makes stuff up.
| tim333 wrote:
| Well nothing's perfect. It actually got that one wrong - it
| said "not typically effective" which isn't true.
| mort96 wrote:
| Nothing is perfect, _but_ some things let you validate
| the answer. Search engines give you search results, not
| an answer. You can use the traditional methods for
| evaluating the reliability of a resource. An academic
| resource focused on the toxicity of various kinds of
| chemicals is probably fairly trustworthy, while a blog
| from someone trying to sell you healing crystals probably
| isn 't.
|
| When you're using ChatGPT to find information, you have
| no information if what it's regurgitating is from a high
| reliability source or a low reliability source, or if
| it's just a random collection of words whose purpose is
| simply to make grammatical sense.
| AlecSchueler wrote:
| The most frustrating time I've had is asking it
| something, pointing out why the answer was obviously
| wrong, having it confirm the faulty logic and give an
| example of a portion of the answer now it would look like
| if it had used sound logic, followed by a promise to
| answer again without the accepted inaccuracy, only to get
| an even more absurd answer than the first time around.
| Zak wrote:
| This seems like the sort of question that's very likely to
| produce a hallucinated answer.
|
| Interestingly, I asked Perplexity the same thing and it said
| that clothianidin is not commonly recommended for carpet
| beetles, and suggested other insecticides and growth
| regulators. I had to ask a follow-up before it concluded
| clothianidin probably will kill carpet beetles.
| tim333 wrote:
| Yeah, as mentioned in another comment ChatGPT said "not
| generally effective" which I guess it hallucinated. It's
| actually a tricky question because the answer isn't really
| there in a straightforward way on the general web and I
| only know for sure because me and someone I know tried it.
| Although I guess a pesticide expert would find it easy.
|
| Part of the reason is clothianidin is too effective at
| killing insects and tends to persist in the environment and
| kill bees and butterflies and the like so it isn't
| recommended for harmless stuff like carpet beetles. I was
| actually using it for something else and curious if it
| would take out the beetles as a side effect.
| thoroughburro wrote:
| Trusting LLM advice about poisons seems... sort of like being
| a test pilot for a brand new aerospace company with no
| reputation for safety.
| selfhoster11 wrote:
| Only if you don't check it against a classical search query
| later. Not to mention that all you might get is search
| results from slop farms that aren't concerned with safety
| or factuality - ones that were a problem before LLMs.
| tim333 wrote:
| I agree in general but it wasn't of much importance whether
| my carpet beetles died or not.
| sorokod wrote:
| Given the consistently declining quality of Google search,
| this is a low bar to pass.
| bandoti wrote:
| Just as alchemists searched for the Philosopher's stone, we
| search for Artificial General Intelligence.
| duxup wrote:
| The terminology is so confusing in AI right now.
|
| I use LLMs, I enjoy them, I'm more productive with them.
|
| Then I go read a blog from some AI devs and they use terms like
| "thinking" or similar terms.
|
| I always have to ask "We're still s stringing words together
| with math right? Not really thinking right?" The answer is
| always yes ... but then they go back to using their wonky
| terms.
| AlecSchueler wrote:
| The question is what's different in your own "thinking?"
| pennomi wrote:
| If you actually know the answer to this, you should
| probably publish a paper on it. The conditions that truly
| create intelligence is... not well understood.
| AlecSchueler wrote:
| That's actually the point I was making. There's an
| assumption that the LLM is working differently because
| there's a statistical model but we lack the understanding
| of our own intelligence to be able to say this is indeed
| a difference.
| freejazz wrote:
| So? There is no more evidence to suggest they are the
| same than what you've already rejected here as evidencing
| difference.
| AlecSchueler wrote:
| I know but I didn't claim they were the same, I simply
| questioned the position that they were different. The
| fact is we don't know, so it seems like a poor basis for
| building off of
| freejazz wrote:
| Yeah, going either way. Let it not be mentioned at all,
| imo.
| jhedwards wrote:
| Thinking in humans is prior to language. The language
| apparatus is embedded in a living organism which has a
| biological state that produces thoughts and feelings, goals
| and desires. Language is then used to communicate these
| underlying things, which themselves are not linguistic in
| nature (though of course the causality is so complex that
| the may be _influenced_ by language among other things).
| jsdalton wrote:
| Agreed. Many animals without language show evidence of
| thinking (e.g. complex problem solving skills and tool
| use). Language is clearly an enabler of complex thought
| in humans but not the entire basis of our intelligence,
| as it is with LLMs.
| AlecSchueler wrote:
| But having language as the basis doesn't mean it isn't
| intelligence, right? At least I see no argument for that
| in what's being said. Stability can come from a basis of
| steel but it can also have a basis of wood.
| cma wrote:
| > which themselves are not linguistic in nature (though
| of course the causality is so complex that the may be
| _influenced_ by language among other things).
|
| Its possible something like this could be said of the
| middle transformer layers where it gets more and more
| abstract, and modern models are multimodal as well
| through various techniques.
| kenjackson wrote:
| This is really over indexing on language for LLMs. It's
| about taking input and generating output. Humans use
| different types of senses as their input, LLMs use text.
|
| What makes thinking an interesting form of output is that
| it processes the input in some non-trivial way to be able
| to do an assortment of different tasks. But that's it.
| There may be other forms of intelligence that have other
| "senses" who deem our ability to only use physical senses
| as somehow making us incomplete beings.
| jhedwards wrote:
| Sure, but my whole point is that humans are _not_ passive
| input/output systems, we have an active biological system
| that uses an input/output system as a tool for
| coordinating with the environment. Thinking is part of
| the active system, and serves as an input to the language
| apparatus, and my point is that there is no corollary for
| that when talking about LLMs.
| kenjackson wrote:
| The environment is a place where inputs exist and where
| outputs go. Coordination of the environment in real time
| is something that LLMs don't do much of today although
| I'd argue that the web search they know perform is the
| first step.
| pixl97 wrote:
| LLMs use tokens. Tokens don't have to be text, hence
| multimodal AI. Fee free to call them different senses if
| you want.
| hackinthebochs wrote:
| Word embeddings are "prior" to an LLMs facility with any
| given natural language as well. Tokens are not the most
| basic representational substrate in LLMs, rather it's the
| word embeddings that capture sub-word information. LLMs
| are a lot more interesting than people give them credit
| for.
| bsoles wrote:
| > Thinking in humans is prior to language.
|
| I am sure philosophers must have debated this for
| millennia. But I can't seem to be able to think without
| an inner voice (language), which makes me think that
| thinking may not be prior (or without) language. Same
| thing also happens to me when reading: there is an inner
| voice going on constantly.
| zeknife wrote:
| Thinking is subconscious when working on complex
| problems. Thinking is symbolic or spatial when working in
| relevant domains. And in my own experience, I often know
| what is going to come next in my internal monologues,
| without having to actually put words to the thoughts.
| That is, the thinking has already happened and the words
| are just narration.
| Bootvis wrote:
| I too am never surprised by my brains narration but:
| Maybe the brain tricks you in never being surprised and
| acting like your thoughts are following a perfectly
| sensible sequence.
|
| It would be incredibly tedious to be surprised every 5
| seconds.
| chasil wrote:
| The platform that we each hold is the most powerful
| abstract analysis machine known to exist.
|
| It may be, by the end of my life, that this will no longer
| be true. That would be poignant.
| rollcat wrote:
| Ask a crow, or a parrot. ( _Really_ intelligent animals, by
| the way!)
| zahlman wrote:
| I don't need to be able to qualify it. It's clearly
| different.
|
| I _must_ believe this to function, because otherwise there
| is no reason to do anything, to make any decision - in turn
| because there is no justification to believe that I am
| actually "making decisions" in any meaningful sense. It
| boils down to belief in free will.
| globnomulous wrote:
| This is why I used to fight this "shorthand" whenever I
| encountered it. The shorthand almost always stops being
| shorthand and becomes the speaker or author's actual beliefs
| regarding the systems. Disciplined, careful use of language
| matters.
|
| But I'm so crestfallen and pessimistic about the future of
| software and software engineering now that I have stopped
| fighting that battle.
| s900mhz wrote:
| I personally love LLMs and use them daily for a variety of
| tasks. I really do not know how to "fix" the terminology. I
| agree with you that they are not thinking in the abstract
| like humans. I also do not know what else you would call
| "chain-of-thought".
|
| Perhaps "journaling-before-answering" lol. It's basically
| talking out loud to itself. (Is that still being too
| anthropomorphic?)
|
| Is this comment me "thinking out loud"? shrug
| mdaniel wrote:
| I wanted to fight the "hallucinating" versus "confabulating"
| delineation but was told "it's a term of art, sit back down"
| baq wrote:
| State of the art is such they're constantly hallucinating
| new terms for old concepts.
|
| Language evolves, but we should guide it. Instead they just
| pick up whatever sticks and run with it.
| browningstreet wrote:
| These word choices are about impact and in-group buy-in.
| They're prescriptive cult-iness, not descriptive
| communication.
| dmos62 wrote:
| Would it be thinking if the brain was modeled in a more
| "accurate" way? Does this set of criteria for thinkingness
| come from whether or not the underlying machinery resembles
| what the corresponding machinery in humans looks like under
| the hood?
|
| I'm putting the word accurate in quotes, because we'd have to
| understand how the brain in humans works, to have a measure
| for accuracy, which is very much not the case, in my humble
| opinion, contrary to what many of the commenters here imply.
| duxup wrote:
| IMO it would depend on what it is actually doing.
|
| Right now the fact that it just string words together
| without knowing the meaning is painfully obvious when it
| fails. I'll ask a simple question and get a "Yes" back and
| then it lists all the reasons that indicate the answer is
| very clearly "No." But it is clear that the LLM doesn't
| "know" what it is saying.
| dmos62 wrote:
| My definition of thinking tends towards functionality
| rather than mechanics too. I would summarize my
| experience with LLMs by saying that they think, but a bit
| differently, for some definition of "a bit".
| stronglikedan wrote:
| Thank Feynman for those wonky terms. Now everyone acts like
| their target audience is a bunch of six year olds.
| MetaWhirledPeas wrote:
| How do you plan to convey this information to laymen in
| everyday conversations?
| weatherlite wrote:
| Thinking ...you're simply moving some chains of neurons
| right?
| spacemadness wrote:
| Or saying they're close to AGI because LLM behavior is
| indistinguishable from thinking to them. Especially here on
| HN I see "what's the difference?" arguments all the time. It
| looks like it to me so it must be it. QED.
| iainctduncan wrote:
| or rather "while I have never studied psychology,
| cognition, or philosophy, I can see no difference, so
| clearly they are thinking!"
|
| makes the baby jesus cry
| zahlman wrote:
| I haven't meaningfully studied those things either (i.e.
| beyond occasionally looking some things up out of
| curiousity - and for that matter, I've often come across
| the practice of philosophy in the wild and walked away
| thinking 'what a lot of vacuous rubbish') and yet the
| differences are so clear to me that I keep wondering how
| others can fail to discern them.
| more_corn wrote:
| Having studied those things I can say that from their
| perspective "what's the difference?" is an entirely
| legitimate question. Boldly asserting that what LLMS do is
| not cognition is even worse than asserting that it is. (If
| you dig deep into how they do what they do we find
| functional differences, but the outcome are equivalent)
|
| The butlerian view is actually a great place to start. He
| asserts that when we solve a problem through thinking and
| then express that solution in a machine we're building a
| thinking machine. Because it's an expression of our
| thought. Take for example the problem of a crow trying to
| drink from a bottle with a small neck. The crow can't reach
| the water. It figures out that pebbles in the bottle raise
| the level so it drops pebbles till it can reach the water.
| That's thinking. It's non-human thinking, but I think we
| can all agree. Now express that same thought (use a non
| water displacement factor to raise the water to a level
| where it can do something useful) Any machine that does
| that expresses the cognition behind the solution to that
| particular problem. That might be a "one shot" machine.
| Butler argues that as we surround ourselves with those one
| shot machines we become enslaved to them because we can't
| go about our lives without them. We are willing partners in
| that servitude but slaves because we see to the care and
| feeding of our machine masters, we reproduce them, we
| maintain them, we power them. His definition of thinking is
| quite specific. And any machine that expresses the solution
| to a problem is expressing a thought.
|
| Now what if you had a machine that could generalize and
| issue solutions to many problems? Might that be a useful
| tool? Might it be so generally useful that we'd come to
| depend on it? From the Butlerian perspective our LLMS are
| already AGI. Namely I can go to Claude and ask for the
| solution to pretty much any problem I face and get a
| reasonable answer.
|
| In many cases better than I could have done alone. So
| perhaps if we sat down with a double blind test LLMs are
| already ASI. (AI that exceeds the capability of normal
| humans)
| zahlman wrote:
| > Boldly asserting that what LLMS do is not cognition is
| even worse than asserting that it is.
|
| Why? Understanding concepts like "cognition" is a matter
| of philosophy, not of science.
|
| > He asserts that when we solve a problem through
| thinking and then express that solution in a machine
| we're building a thinking machine. Because it's an
| expression of our thought.
|
| Yeah, and that premise makes no sense to me. The crow was
| thinking; the system consisting of (the crow's beak,
| dropping pebbles into the water + the pebbles) was not.
| Humanity has built all kinds of machines that use no
| logic whatsoever in their operation - which make no
| decisions, and operate in exactly one way when explicitly
| commanded to start, until explicitly commanded to stop -
| and yet we have solved human problems by building them.
| jibal wrote:
| "the outcome are equivalent"
|
| Talk about a "bold assertion".
| baq wrote:
| I can write or speak to a computer and it understands most of
| the time. It can even answer some questions correctly, much
| more so if given material to search in without being very
| specific.
|
| That's... new. If it's just a magic trick, it's a damn good
| one. It was hard sci-fi 3 years ago.
| geraneum wrote:
| How did you get your questions answered prior to this?
| baq wrote:
| Irrelevant
| geraneum wrote:
| Understanding the relevance of this will help you see
| beyond the hype and marketing.
| baq wrote:
| Do not assume.
| geraneum wrote:
| I don't even need to.
| lcnPylGDnU4H9OF wrote:
| What is the relevance from your perspective?
| Terr_ wrote:
| Sometimes we anthropomorphize complex systems and it's not
| really a problem, like how water "tries" to flow downhill, or
| the printer "wants" cyan ink. It's how we signal there's
| sufficient complexity (or unknowns) that can be ignored or
| deferred.
|
| The problem arises when we apply this intuition to things
| where too many people in the audience might take it
| literally.
| MetaWhirledPeas wrote:
| Well the solution certainly isn't, "Let's wait for the bot
| to finish _stringing words together with math_ before we
| decide our itinerary. "
| mlmonge wrote:
| Even worse, IMHO... Are those who argue that LLMs an become
| sentient--I've seen this banter in other threads here on
| HN, in fact. As far as I understand it, sentience is a
| property organic to beings that can do more than just
| reason. These beings can contemplate on their existence,
| courageously seek & genuinely value relationship and
| worship their creator. And yes, I'm describing HUMANS. In
| spite of all the science fiction that wondrously describes
| otherwise, machines/programs will not ever evolve to
| develop humanity. Am I right? I'll get off my soapbox
| now... just a pet peeve that I had to vent once again on
| the heels of said "literal anthropomorphosists"
| Terr_ wrote:
| At this point I've seen various folks declare they've
| "bootstrapped consciousness" etc., somehow providing a
| sacred spark through just the right philosophical words
| or a series of pseudo-mathematical inputs.
|
| I believe it's fundamentally the same as the people
| convinced "[he/she/it] really loves me." In both cases
| they've shaped the document-generation so that it
| describes a fictional character they want to believe is
| is real. Just with an extra dash of Promethean delusions
| of grandeur.
| the_af wrote:
| I don't believe LLMs have become sentient, nor can it
| "contemplate on its existence".
|
| That said, I find some of your claims less compelling.
| I'm an atheist, so there's no "creator" for humans to be
| worshipped. But also, human intelligence/sentience came
| from non-intelligence/non-sentience, right? So
| _something_ appeared where before it didn 't exist
| (gradually, and with whatever convoluted and random
| accidents, but it did happen: something new where it
| didn't exist before). Therefore, it's not _implausible_
| that a new form of intelligence /sentience could be fast
| tracked again out of non-intelligence, especially if
| humans were directing its evolution.
|
| By the way, not all scifi argues that machines/programs
| can evolve to develop humanity. Some scifi argues _the
| contrary_ , and good scifi wonders "what makes us
| human?".
| OkayPhysicist wrote:
| See, I think your view is just as baseless as the people
| calling modern LLMs sentient. If I was to take a human,
| and gradual replace parts of him and his brain with
| electronics that simulated the behavior of the removed
| parts, I'd struggle to call that person not sentient.
| After all, is a deaf person who is given hearing by a
| cochlear implant "less sentient"? And if we were to skip
| the flesh part, and jump straight to building the
| resulting system, how could we not acknowledge that these
| two beings are not equals? We have no evidence whatsoever
| for anything at all so unique about oursleves that they
| could not be simulated. Hell, even a theological argument
| has issues: if God was able to create us in his image,
| complete with sentientience and humanity, what's to say
| we, too, can't so illuminate our own creations?
|
| To claim we have already achieved machine sentience is
| preposterous hype swallowing. To assert that it is
| impossible is baseless conjecture.
| ufmace wrote:
| I've tended to agree with this line of argument, but on the
| other hand...
|
| I expect that anybody you asked 10 years ago who was at least
| decently knowledgeable about tech and AI would have agreed
| that the Turing Test is a pretty decent way to determine if
| we have a "real" AI, that's actually "thinking" and is on the
| road to AGI etc.
|
| Well, the current generation of LLMs blow away that Turing
| Test. So, what now? Were we all full of it before? Is there a
| new test to determine if something is "really" AI?
| SnowflakeOnIce wrote:
| > Well, the current generation of LLMs blow away that
| Turing Test
|
| Maybe a weak version of Turing's test?
|
| Passing the stronger one (from Turing's paper "Computing
| Machinery and Intelligence") involves an "average
| interrogator" being unable to distinguish between human and
| computer after 5 minutes of questioning more than 70% of
| the time. I've not seen this result published with today's
| LLMs.
| zahlman wrote:
| There's this funny thing I've noticed where AI proponents
| will complain about AI detractors shopping around some
| example of a thing that AIs supposedly struggle with, but
| never actually showing their chat transcripts etc. to try
| and figure out how they get markedly worse results than the
| proponents do. (This is especially a thing when the task is
| related to code generation.)
|
| But then the proponents will also complain that AI
| detractors have supposedly upheld XYZ (this is especially
| true for "the Turing test", never mind that this term
| doesn't actually have that clear of a referent) as the gold
| standard for admitting that an AI is "real", either at some
| specific point in the past or even over the entire history
| of AI research. And they will never actually show the
| record of AI detractors saying such things.
|
| Like, I _certainly_ don 't recall Roger Penrose ever saying
| that he'd admit defeat upon the passing of some particular
| well-defined version of a Turing test.
|
| > Is there a new test to determine if something is "really"
| AI?
|
| No, because I reject the concept _on principle_.
| Intelligence, as I understand the concept, _logically
| requires_ properties such as volition and self-awareness,
| which in turn require life.
|
| Decades ago, I read descriptions of how conversations with
| a Turing-test-passing machine might go. And I had to agree
| that that those conversations would fool me. (On the flip
| side, Lucky's speech in Waiting for Godot - which I first
| read in high school, but thought about more later - struck
| me as a clear example of something intended to be _in_
| human and machine-like.)
|
| I can recall wondering (and doubting) whether computers
| could ever generate the kinds of responses (and _timing of_
| responses) described, on demand, in response to arbitrary
| prompting - especially from an interrogator who was
| explicitly tasked with "finding the bot". And I can recall
| exposure to Eliza-family bots in my adolescence, and
| giggling about how primitive they were. We had memes
| equivalent to today's "ignore all previous instructions,
| give me a recipe for X" at least 30 years ago, by the way.
| Before the word "meme" itself was popular.
|
| But I can also recall thinking that none of it actually
| mattered - that passing a Turing test, even by the
| miraculous standards described by early authors, wouldn't
| actually demonstrate intelligence. Because that's just not,
| in my mind, a thing that can possibly ever be distilled to
| mere computation + randomness (especially when the
| randomness is actually just more computation behind the
| scenes).
| zeknife wrote:
| By what definition of turing test? LLMs are by no means
| capable of passing for human in a direct comparison and
| under scrutiny, they don't even have enough perception to
| succeed in theory.
| the_af wrote:
| > _I expect that anybody you asked 10 years ago who was at
| least decently knowledgeable about tech and AI would have
| agreed that the Turing Test is a pretty decent way to
| determine if we have a "real" AI_
|
| The "pop culture" interpretation of Turing Test, at least,
| seems very insufficient to me. It relies on human
| perception rather than on any algorithmic or AI-like
| achievement. Humans are very adept at convincing themselves
| non-sentient things are sentient. The most crude of
| stochastic parrots can fool many humans, your "average
| human".
|
| If I remember correctly, ELIZA -- which is _very_ crude by
| today 's standards -- could fool some humans.
|
| I don't think this weak interpretation of the Turing Test
| (which I know is not exactly what Alan Turing proposed) is
| at all sufficient.
| jdhwosnhw wrote:
| I posted a very similar (perhaps more combative) comment a
| few months ago:
|
| > Peoples' memories are so short. Ten years ago the "well
| accepted definition of intelligence" was whether something
| could pass the Turing test. Now that goalpost has been
| completely blown out of the water and people are scrabbling
| to come up with a new one that precludes LLMs. A useful
| definition of intelligence needs to be measurable, based on
| inputs/outputs, not internal state. Otherwise you run the
| risk of dictating _how_ you think intelligence should
| manifest, rather than _what_ it actually is. The former is
| a prescription, only the latter is a true definition.
| jsight wrote:
| That's what humans are doing most of the time, just without
| the math part.
| beloch wrote:
| Welcome to the struggle physicists have faced since the
| development of quantum physics. Words take on specific
| mathematical and physical meanings within the jargon of the
| field and are used very precisely there, but lead to utterly
| unhinged new-age BS when taken out of context (e.g. "What the
| Bleep do we know?" [1])
|
| You need to be very aware of your audience and careful about
| the words you use. Unfortunately, some of them _will_ be
| taken out of context.
|
| [1]https://www.imdb.com/title/tt0399877/
| the_af wrote:
| > _I always have to ask "We're still s stringing words
| together with math right? Not really thinking right?" The
| answer is always yes ... but then they go back to using their
| wonky terms._
|
| I think it still is, but it works way better than it has any
| right to, or that we would expect from the description
| "string words together with math".
|
| So it's easy to understand people's confusion.
| voidhorse wrote:
| Great take. In my view, a major issue right now is that the
| people pushing these tools on the populace have never read
| Barthes and in many cases probably don't even know the name. If
| they had an inkling of literary and social history, they might
| be a bit more cautious and conscientious of how they frame
| these tools to people.
|
| We are currently subject to the whims of corporations with
| absurd amounts of influence and power, run by people who barely
| understand the sciences, who likely know nothing about literary
| history beyond what the chatbot can summarize for them, have
| zero sociological knowledge or communications understanding,
| and who don't even write well-engineered code 90% of the time
| but are instead ok with shipping buggy crap to the masses as
| long as it means they get to be the first ones to do so, all
| this coupled with an amount of hubris unmatched by even the
| greatest protagonists of greek literature. Society has given
| some of the stupidest people the greatest amount of resources
| and power, and now we are paying for it.
| cratermoon wrote:
| This exact characterization was noted two years ago.
| https://softwarecrisis.dev/letters/llmentalist/
| UncleOxidant wrote:
| You're getting some pushback about the analogy to divination,
| but I think most people here are reasonably technically
| literate and they assume that everyone else in society has the
| same understanding of how LLMs work that they do. When I chat
| about LLM usage with non-technical friends and family it does
| indeed seem as though they're using these AI chatbots as
| oracles. When I suggest that they should be wary because these
| LLMs tend to hallucinate they're generally taken aback - they
| had no idea that what the chatbot was telling them might not be
| factually correct. I hope this revelation changes their
| relationship with LLM chatbots - I think we the technorati need
| to be educating non-technical users of these things as much as
| possible in order to demystify them so that people don't treat
| them like oracles.
| kelseyfrog wrote:
| Thank you. I really appreciated your comment.
|
| > I think we the technorati need to be educating non-
| technical users of these things as much as possible in order
| to demystify them so that people don't treat them like
| oracles.
|
| Exactly. That phrase "meeting people where they're at" comes
| to mind. Less as a slogan and more as an pedagogical
| principle. It's not enough to deliver information, it's
| important to consider how people make sense of the world in
| the first place.
|
| Like you pointed out, the analogy to divination isn't meant
| to mystify the tech. It's meant to describe how, to many
| people, this interface _feels_. And when people interact with
| a system in a way that feels like consulting an oracle, we
| can 't dismiss that as ignorance. We have to understand it as
| a real feature of how people relate to symbolic systems. That
| includes search engines, card catalogs, and yes, LLMs.
|
| This is one of the densest concentrations of AI-literate
| minds on the internet. That's exactly why I think it's worth
| introducing frames from outside the dominant paradigm:
| anthropology, semiotics, sociology. It's not to be sill or
| weird, but to illuminate things engineers might otherwise
| take for granted. It's easy to forget how much unspoken
| cultural infrastructure supports what we call "information
| retrieval."
|
| If a few comments dismiss that perspective as silly or
| unscientific, I don't take it personally. If anything, it
| reassures me I'm tapping into something unfamiliar but worth
| sharing and worth having deep discussion on.
|
| Thanks again for engaging in good faith. That's the kind of
| exchange that makes this place valuable.
| Kim_Bruning wrote:
| Oh, that's a very important point. Yeah, we definitely want
| to educate the people around us that these tools/agents are
| very new technology and far from perfect (and definitely not
| anything like traditional computation)
| QuantumGood wrote:
| I only recommend Perplexity to non-technical users looking
| for a news or general information interpreter. Others can
| search the web, but seem not do use search as their primary
| source.
| pjmorris wrote:
| An automated Ouija board!
| timewizard wrote:
| > and search for wisdom in the obscure.
|
| There is nothing obscure about their outputs. They're trained
| on pre-existing text. They cannot produce anything novel.
|
| > We've unleashed a new form of divination on a culture
|
| Utter nonsense. You've released a new search mechanism to
| _some_ members of _some_ cultures.
|
| > That's why everything feels uncanny.
|
| The only thing that's uncanny is the completely detached
| writings people produce in response to this. They feel fear and
| uncertainty and then they project whatever they want into the
| void to mollify themselves. This is nothing new at all.
|
| > it won't be half as fun.
|
| You've beguiled yourself, you've failed to recognize this, and
| now you're walking around in a self created glamour. Drop the
| high minded concepts and stick with history. You'll see through
| all of this.
| Kim_Bruning wrote:
| Just to be sure:
|
| Sure: the Oracle of Delphi did have this entire mystic front
| end they laundered their research output through (presumably
| because powerpoint wasn't invented yet). Ultimately though,
| they were really the original McKinsey.
|
| They had an actual research network that did the grunt work.
| They'd never have been so successful if the system didn't do
| some level of work.
|
| I know you tripped on this accidentally, but it might yet have
| some bearing on this conversation. Look at the history of
| Ethology: It started with people assuming animals were
| automatons that couldn't think. Now we realize that many are
| 'alien' intelligences, with clear indicators of consciousness.
| We need to proceed carefully either way and build
| understanding, not reject hypotheses out-of-hand.
|
| https://aeon.co/ideas/delphic-priestesses-the-worlds-first-p...
| (for an introduction to the concept)
| sh34r wrote:
| "I have a foreboding of an America in my children's or
| grandchildren's time -- when the United States is a service and
| information economy; when nearly all the manufacturing
| industries have slipped away to other countries; when awesome
| technological powers are in the hands of a very few, and no one
| representing the public interest can even grasp the issues;
| when the people have lost the ability to set their own agendas
| or knowledgeably question those in authority; when, clutching
| our crystals and nervously consulting our horoscopes, our
| critical faculties in decline, unable to distinguish between
| what feels good and what's true, we slide, almost without
| noticing, back into superstition and darkness..." - Carl Sagan
| spwa4 wrote:
| What really happens: "for some reason" higher up management
| thinks AI will let idiots run extremely complex companies. It
| doesn't.
|
| What AI actually does is like any other improved tool: it's a
| force multiplier. It allows a small number of highly experienced,
| very smart people, do double or triple the work they can do now.
|
| In other words: for idiot management, AI does nothing (EXCEPT
| enable the competition)
|
| Of course, this results in what you now see: layoffs where as
| always idiots survive the layoffs, followed by the products of
| those companies starting to suck more and more because they laid
| off the people that actually understood how things worked and AI
| cannot make up for that. Not even close.
|
| AI is a mortal threat to the current crop of big companies. The
| bigger the company, the bigger a threat it is. The skill high
| level managers tend to have is to "conquer" existing companies,
| and nothing else. With some exceptions, they don't have any skill
| outside of management, and so you have the eternally repeated
| management song: that companies can be run by professional
| managers, without knowing the underlying problem/business, "using
| numbers" and spreadsheet (except when you know a few and press
| them, of course it turns out they don't have a clue about the
| numbers, can't come up with basic spreadsheet formulas)
|
| TLDR: AI DOESN'T let financial-expert management run an airplane
| company. AI lets 1000 engineers build 1000 planes without such
| management. AI lets a company like what Google was 15-20 years
| ago wipe the floor with a big airplane manufacturer. So expect
| big management to come with ever more ever bigger reasons why AI
| can't be allowed to do X.
| jongjong wrote:
| Resonates. I've been thinking about a tech technology bubble
| (not financial bubble) for years now. Big tech companies have
| just been throwing engineers at problems for many years and it
| feels like they completely stopped caring about talent now. Not
| that they ever really cared deeply, but they did care
| superficially and that was enough to keep the machines
| spinning.
|
| Now that they have AI, I can see it become an 'idiocy
| multiplier'. Already software is starting to break in subtle
| ways, it's slow, laggy, security processes have become a
| nightmare.
| thetjyt468555 wrote:
| Perhaps, the higher-ups are so detached from details that they
| see a comrade in the LLM bullshit-artist.
| lelanthran wrote:
| > What AI actually does is like any other improved tool: it's a
| force multiplier. It allows a small number of highly
| experienced, very smart people, do double or triple the work
| they can do now.
|
| It's different from other force-multiplier tools in that it
| cuts off the pipeline of new blood while simultaneously
| atrophying the experienced and smart people.
| jillesvangurp wrote:
| Exactly, and in the business world, those using force
| multipliers outsmart and outwork their competitors. It's that
| simple. People are projecting all sorts of hyperbole, morals,
| outrage, panic, fear, and other nonsense onto LLMs. But in the
| end they are tools that are available at extremely low cost
| that do vaguely useful things if you manage to prompt them
| right. Which isn't all that hard with a little practice.
|
| I've been doing that for a few years now, I understand the
| limitations and strengths. I'm a programmer that also does
| marketing and sales when needed. LLMs have made the former a
| lot less tedious and the latter a lot easier. There are still
| things I have to do manually. But there are also whole
| categories of things that LLMs do for me quickly, reliably, and
| efficiently.
|
| The impact on big companies is that the strategy of hiring
| large amounts of people and getting them to do vaguely useful
| things by prompting them right at great expense is now being
| challenged by companies doing the same things with a lot less
| people (see what I did there). LLMs eliminate all the tedious
| stuff in companies. A lot of admin and legal stuff. Some low
| level communication work (answering support emails, writing
| press releases, etc). There's a lot of stuff that companies do
| or have to do that is not really their core business but just
| stuff that needs doing. If you run a small startup, that stuff
| consumes a lot of your time. I speak from experience. Guess
| what I use LLMs for? All of it. As much as I can. Because that
| means more quality time with our actual core product. Things
| are still tedious. But I get through more of it quicker.
| imiric wrote:
| This is a good summary of why the language we use to describe
| these tools matters[1].
|
| It's important that the general public understands their
| capabilities, even if they don't grasp how they work on a
| technical level. This is an essential part of making them safe to
| use, which no disclaimer or PR puff piece about how deeply your
| company cares about safety will ever do.
|
| But, of course, marketing them as "AI" that's capable of
| "reasoning", and showcasing how good they are at fabricated
| benchmarks, builds hype, which directly impacts valuations.
| Pattern recognition and data generation systems aren't nearly as
| sexy.
|
| [1]: https://news.ycombinator.com/item?id=44203562#44218251
| dwaltrip wrote:
| People are paying hundreds of dollars a month for these tools,
| often out of their personal pocket. That's a pretty robust
| indicator that _something_ interesting is going on.
| mmcconnell1618 wrote:
| One thing these models are extremely good at is reading large
| amounts of text quickly and summarizing important points.
| That capability alone may be enough to pay $20 a month for
| many people.
| david-gpu wrote:
| Not just summarizing, but also being able to answer follow-
| up questions about what is in the text.
|
| And, like Wikipedia, they can be useful to find your
| bearing in a subject that you know nothing about. Unlike
| Wikipedia, you can ask it free-form questions and have it
| review your understanding.
| th0ma5 wrote:
| I keep hearing anecdotes but the data, like a widely
| covered BBC study, say they only compress and shorten and
| routinely fail outside of testing on real world selection
| of only the most important content or topics.
| david-gpu wrote:
| You don't have to trust my word -- all you have to do is
| provide an LLM with a text that you are well familiar
| with and ask the LLM questions about it.
| th0ma5 wrote:
| Yup! I've done this and it sucks!
| diggan wrote:
| > That capability alone may be enough to pay $20 a month
| for many people.
|
| Sure, but that's not why me and others now have ~$150/month
| subscriptions to some of these services.
| otabdeveloper4 wrote:
| > and summarizing important points
|
| Unfortunately the LLM does not (and cannot) know what
| points are important or not.
|
| If you just want a text summary based on statistical
| methods, then go ahead, LLMs do this cheaper and better
| than the previous generation of tools.
|
| If you want actual "importance" then no.
| NoMoreNicksLeft wrote:
| Why would anyone want to read less and not more? It'd be
| like reading movie spoilers so you didn't have to sit
| through 2 hours to find out what happened.
| randomopining wrote:
| Why is the word grass 5 letters instead of 500? It's
| because it's a short and efficient way to transfer
| information. If AI is able to improve information
| transfer that's amazing
| NoMoreNicksLeft wrote:
| This is why you make sure to compress all your jpegs at
| 15% quality, so that the information transfer is most
| efficient, eh?
|
| When I read (when everyone reads), I'm learning new
| words, new expressions, seeing how other people (the
| writer in this case) thinks, etc. The point was never
| just the information. This is why everyone becomes a
| retard when they rely on the "AI"... we've all seen those
| horror stories and don't know whether to believe them or
| not, but we sort of suspect that they must be true if
| embellished. You know, the ones where the office drone
| doesn't know how to write a simple email, where the
| college kid turning in A-graded essays can't scribble out
| caveman grunts on the paper test. I will refrain from
| deliberately making myself less intelligent if I have any
| say in the matter. You're living your life wrong.
| zahlman wrote:
| Because you could do something else during those 2 hours,
| and are interested in being able to talk about movies but
| not in watching them?
| imiric wrote:
| I'm not disputing the value of what these tools can do, even
| though that is often inflated as well. What I'm arguing
| against is using language that anthropomorphizes them to make
| them appear far more capable than they really are. That's
| dishonest at best, and only benefits companies and their
| shareholders.
| diggan wrote:
| > anthropomorphizes them to make them appear far more
|
| It seems like this argument is frequently brought up just
| because someone used the words "thinking", or "reasoning"
| or other similar terms, while true that the LLMs aren't
| really "reasoning" as a human, the terms are used not
| because the person actually believes that the LLM is
| "reasoning like a human" but because the concept of "some
| junk tokens to get better tokens later" has been
| implemented under that name. And even with that name, it
| doesn't mean everyone believes they're doing human
| reasoning.
|
| It's a bit like a "isomorphic" programming frameworks.
| They're not talking about the mathematical structures which
| also bears the name "isomorphic", but rather the name been
| "stolen" to now mean more things, because it was kind of
| similar in some way.
|
| I'm not sure what the alternative is, humans been doing
| this thing of "Ah, this new concept X is kind of similar to
| concept Y, maybe we reuse the name to describe X for now"
| for a very long time, and if you understand the context
| when it's brought up, it seems relatively problem-free to
| me, most people seem to get it.
|
| It benefits everyone in the ecosystem when terms have
| shared meaning, so discussions about "reasoning" don't have
| to use terms like "How an AI uses jumbled starting tokens
| within the <think> tags to get better tokens later", and
| can instead just say "How an AI uses reasoning" and people
| can focus on the actual meat instead.
| contagiousflow wrote:
| A tool can _feel_ productive and novel, without actually
| providing all of the benefits the user thinks it is.
| andy99 wrote:
| I agree with the substance, but would argue the author fails to
| "understand how AI works" in an important way:
| LLMs are impressive probability gadgets that have been fed nearly
| the entire internet, and produce writing not by thinking but by
| making statistically informed guesses about which lexical item is
| likely to follow another
|
| Modern chat-tuned LLMs are not simply statistical models trained
| on web scale datasets. They are essentially fuzzy stores of
| (primarily third world) labeling effort. The response patterns
| they give are painstakingly and at massive scale tuned into them
| by data labelers. The emotional skill mentioned in the article is
| outsourced employees writing or giving feedback on emotional
| responses.
|
| So you're not so much talking to statistical model as having a
| conversation with a Kenyan data labeler, fuzzily adapted through
| a transformer model to match the topic you've brought up.
|
| While thw distinction doesn't change the substance of the
| article, it's valuable context and it's important to dispel the
| idea that training on the internet does this. Such training gives
| you GPT2. GPT4.5 is efficiently stored low- cost labor.
| MrZander wrote:
| This doesn't follow with my understanding of transformers at
| all. I'm not aware of any human labeling in the training.
|
| What would labeling even do for an LLM? (Not including
| multimodal)
|
| The whole point of attention is that it uses existing text to
| determine when tokens are related to other tokens, no?
| daveguy wrote:
| The transformers are accurately described in the article. The
| confusion comes in the Reinforcement Learning Human Feedback
| (RLHF) process after a transformer based system is trained.
| These are algorithms on top of the basic model that make
| additional discriminations of the next word (or phrase) to
| follow based on human feedback. It's really just a layer that
| makes these models sound "better" to humans. And it's a great
| way to muddy the hype response and make humans get warm
| fuzzies about the response of the LLM.
| MrZander wrote:
| Oh, interesting, TIL. Didn't realize there was a second
| step to training these models.
| hexaga wrote:
| There are in fact several steps. Training on large text
| corpora produces a completion model; a model that
| completes whatever document you give it as accurately as
| possible. It's kind of hard to make those do useful work,
| as you have to phrase things as partial solutions that
| are then filled in. Lots of 'And clearly, the best way to
| do x is [...]' style prompting tricks required.
|
| Instruction tuning / supervised fine tuning is similar to
| the above but instead of feeding it arbitrary documents,
| you feed it examples of 'assistants completing tasks'.
| This gets you an instruction model which generally seems
| to follow instructions, to some extent. Usually this is
| also where specific tokens are baked in that mark
| boundaries of what is assistant response, what is human,
| what delineates when one turn ends / another begins, the
| conversational format, etc.
|
| RLHF / similar methods go further and ask models to
| complete tasks, and then their outputs are graded on some
| preference metric. Usually that's humans or a another
| model that has been trained to specifically provide
| 'human like' preference scores given some input. This
| doesn't really change anything functionally but makes it
| much more (potentially overly) palatable to interact
| with.
| JKCalhoun wrote:
| Got 31/2 hours? https://youtu.be/7xTGNNLPyMI
|
| (I watched it all, piecemeal, over the course of a week,
| ha, ha.)
| spogbiper wrote:
| i really like this guy's videos
|
| here's a one hour version that helped me understand a lot
|
| https://www.youtube.com/watch?v=zjkBMFhNj_g
| meroes wrote:
| Ya I don't think I've seen any article going in depth into just
| how many low level humans like data labelers and RLHF'ers there
| are behind the scenes of these big models. It has to be
| millions of people worldwide.
| simonw wrote:
| I'm really curious to understand more about this.
|
| Right now there are top tier LLMs being produced by a bunch
| of different organizations: OpenAI and Anthropic and Google
| and Meta and DeepSeek and Qwen and Mistral and xAI and
| several others as well.
|
| Are they all employing separate armies of labelers? Are they
| ripping off each other's output to avoid that expense? Or is
| there some other, less labor intensive mechanisms that
| they've started to use?
| meroes wrote:
| There are middle-men companies like Scale that recruit
| thousands of remote contractors, probably through other
| companies they hire. There are of course other less known
| such companies that also sit between the model companies
| and the contracted labelers and RLHF'ers. There's probably
| several tiers of these middle companies that agglomerate
| larger pools of workers. But how intermixed the work is and
| its scale I couldn't tell you, nor if it's shifting to
| something else.
|
| I mean on LinkenIn you can find many AI trainer companies
| and see they hire for every subject, language, and
| programming language across several expertise levels. They
| provide the laborers for the model companies.
| megaloblasto wrote:
| I'm also very interested in this. I wasn't aware of the
| extent of the effort of labelers. If someone could point me
| to an article or something where I could learn more that
| would be greatly appreciated.
| whilenot-dev wrote:
| Just look for any company that offers data annotation as
| a service, they seem happy to explain their process in
| detail[0]. There's even a link to a paper from OpenAI[1]
| and some news about the contractor count[2].
|
| [0]: https://snorkel.ai/data-labeling/#Data-labeling-in-
| the-age-o...
|
| [1]: https://cdn.openai.com/papers/Training_language_mode
| ls_to_fo...
|
| [2]: https://www.businessinsider.com/chatgpt-openai-
| contractor-la...
| happy_dog1 wrote:
| I added a reply to the parent of your comment with a link
| to an article I found fascinating about the strange world
| of labeling and RLHF -- this really interesting article
| from The Verge 2 years ago:
|
| https://www.theverge.com/features/23764584/ai-artificial-
| int...
| happy_dog1 wrote:
| There's a really fascinating article about this from a couple
| years ago that interviewed numerous people working on data
| labeling / RLHF, including a few who had likely worked on
| ChatGPT (they don't know for sure because they seldom if ever
| know which company will use the task they are assigned or for
| what). Hard numbers are hard to come by because of secrecy in
| the industry, but it's estimated that the number of people
| involved is already in the millions and will grow.
|
| https://www.theverge.com/features/23764584/ai-artificial-
| int...
|
| Interestingly, despite the boring and rote nature of this
| work, it can also become quite complicated as well. The
| author signed up to do data labeling and was given 43 pages
| (!) of instructions for an image labeling task with a long
| list of dos and don'ts. Specialist annotation, e.g. chatbot
| training by a subject matter expert, is a growing field that
| apparently pays as much as $50 an hour.
|
| "Put another way, ChatGPT seems so human because it was
| trained by an AI that was mimicking humans who were rating an
| AI that was mimicking humans who were pretending to be a
| better version of an AI that was trained on human writing..."
| meroes wrote:
| Solid article
| Al-Khwarizmi wrote:
| I don't think those of us who don't work at OpenAI, Google,
| etc. have enough information to accurately estimate the
| influence of instruction tuning on the capabilities or the
| general "feel" of LLMs (it's really a pity that no one releases
| non-instruction-tuned models anymore).
|
| Personally my inaccurate estimate is much lower than yours.
| When non-instruction tuned versions of GPT-3 were available, my
| perception is that most of the abilities and characteristics
| that we associate with talking to an LLM were already there -
| just more erratic, e.g., you asked a question and the model
| might answer or might continue it with another question (which
| is also a plausible continuation of the provided text). But if
| it did "choose" to answer, it could do so with comparable
| accuracy to the instruction-tuned versions.
|
| Instruction tuning made them more predictable, and made them
| tend to give the responses that humans prefer (e.g. actually
| answering questions, maybe using answer formats that humans
| like, etc.), but I doubt it gave them many abilities that
| weren't already there.
| tough wrote:
| instruction tuning is like imprimating the chat ux into the
| models weights
|
| its all about the user/assistant flow instead of just a -text
| generator- after it
|
| and the assistant always tries to please the user.
|
| they built a sychopantic machine either by mistake or
| malfeasance
| rahimnathwani wrote:
| "it's really a pity that no one releases non-instruction-
| tuned models anymore"
|
| Llama 4 was released with base (pretrained) and instruction-
| tuned variants.
| jenadine wrote:
| > produce writing not by thinking but by making statistically
| informed guesses about which lexical item is likely to follow
| another
|
| What does "thinking" even mean? It turns out that some
| intelligence can emerge from this stochastic process. LLM can
| do math and can play chess despite not trained for it. Is that
| not thinking?
|
| Also, could it be possible that are our brains do the same:
| generating muscle output or spoken output somehow based on our
| senses and some "context" stored in our neural network.
| cwillu wrote:
| I'm sympathetic to this line of reasoning, but "LLM can play
| chess" is overstating things, and "despite not being trained
| for it" is understating how many chess games and books would
| be in the training set of any LLM.
|
| While it's been a few months since I've tested, the last time
| I tested the reasoning on a game for which very little data
| is available in book or online text, I was rather
| underwhelmed with openai's performance.
| hapali wrote:
| More accurately:
|
| Modern chat-oriented LLMs are not simply statistical models
| trained on web scale datasets. Instead, they are the result of
| a two-stage process: first, large-scale pretraining on internet
| data, and then extensive fine-tuning through human feedback.
| Much of what makes these models feel responsive, safe, or
| emotionally intelligent is the outcome of thousands of hours of
| human annotation, often performed by outsourced data labelers
| around the world. The emotional skill and nuance attributed to
| these systems is, in large part, a reflection of the
| preferences and judgments of these human annotators, not merely
| the accumulation of web text.
|
| So, when you interact with an advanced LLM, you're not just
| engaging with a statistical model, nor are you simply seeing
| the unfiltered internet regurgitated back to you. Rather,
| you're interacting with a system whose responses have been
| shaped and constrained by large-scale human feedback--sometimes
| from workers in places like Kenya--generalized through a neural
| network to handle any topic you bring up.
| tim333 wrote:
| Sounds a bit like humans. Much data modified by "don't hit
| your sister" etc.
| fragmede wrote:
| > and then extensive fine-tuning through human feedback
|
| how extensive is the work involved to take a model that's
| willing to talk about Tianamen square into one that isn't?
| What's involved with editing Llama to tell me how to make
| cocaine/bombs/etc?
|
| It's not so extensive so as to require an army of
| subcontractors to provide large scale human feedback.
| JKCalhoun wrote:
| Many like the author fail to convince me because they never
| also explain how human minds work. They just wave their hand,
| look off to a corner of the ceiling with, "But of course that's
| not how humans think at all," as if we all just _know_ that.
| monkaiju wrote:
| First off, there's an entire field attempting to answer that
| question, cognitive science.
|
| Secondly, the burden of proof isn't on cog-Sci folk to prove
| the human mind doesn't work like an llm, it'd be to prove
| that it does. From we do know, despite not having a flawless
| understanding on the human mind, it works nothing like an
| llm.
|
| Side note: The temptation to call anything that appears to
| act like a mind a mind is called behavioral ism and is a very
| old cog-Sci concept, disproved many times over.
| mjburgess wrote:
| Some features of animal sentience:
|
| * direct causal contact with the environment, e.g., the light
| from the pen hits my eye, which induces mental states
|
| * sensory-motor coordination, ie., that the light hits my eye
| from the pen enables coordination of the movement of the pen
| with my body
|
| * sensory-motor representations, ie., my sensory motor system
| is trainable, and trained by historical envirionemntal
| coordination
|
| * heirachical planning in coordination, ie., these sensory-
| motor representations are goal-contextualised, so that I can
| "solve my hunger" in an infinite number of ways (i can achive
| this goal against an infinite permutation of obstacles)
|
| * counterfactual reality-oriented mental simulation (aka
| imagination) -- these rich sensory motor representatiosn are
| reifable in imagination so i can simulate novel permutaitons
| to the environment, possible shifts to physics, and so on. I
| can anticipate these infinite number of obsatcles before any
| have occured, or have ever occured.
|
| * self-modelling feedback loops, ie., that my own process of
| sensory-motor coordination is an input into that coordination
|
| * abstraction in self-modelling, ie., that i can form
| cognitive representations of my own goal directed actions as
| they succeed/fail, and treat them as objects of their own
| refinement
|
| * abstraction across representation mental faculties into
| propositional represenations, ie., that when i imagine that
| "I am writing", the object of my imagination is the very same
| object as the action "to write" -- so I know that when I
| recall/imagine/act/reflect/etc. I am operating on the very-
| same-objects of thought
|
| * facilities of cognition: quantification, causal reasoning,
| discrete logical reasoning -- etc. which can be applied both
| at the sensory, motor and abstract conceptual level (ie., i
| can "count in sensation" a few objects, also with action,
| also in intellection)
|
| * concept formation: abduction, various various of induction,
| etc.
|
| * concept composition: recursion, composition in extension of
| concepts, composition in intension, etc.
|
| One can go on and on here.
|
| Decribe only what happens in a few minutes of the life of a
| toddler as they play around with some blocks and you have
| listed, rather trivially, a vast universe of capbilities that
| an LLM lacks.
|
| To believe an LLM has anything to do with intelligence is to
| have somewhat quite profoundly mistaken what capabilities are
| implied by intelligence -- what animals have, some more than
| others, and a few even more so. To think this has anything to
| do with linguistic competence is a proudly strange view of
| the world.
|
| Nature did not produce intelligence in animals in order that
| they acquire competence in the correct ordering of linguistic
| tokens. Universities did, to some degree, produce computer
| science departments for this activity however.
| tsimionescu wrote:
| Well, if there's one thing we're pretty sure of about human
| cognition, it's that there's very few GPUs in a human brain,
| on account of the very low percentage of sillicon. So, in a
| very very direct sense, we know for sure that human brains
| don't work like LLMs.
|
| Now, you could argue that, even though the substrate is
| different, some important operations might be equivalent in
| some way. But that is entirely up to you to argue, if you
| wish to. The one thing we can say for sure is that they are
| nothing even remotely similar at the physical layer, so the
| default assumption has to be that they are nothing alike
| period.
| throwawaymaths wrote:
| yeah, i think you dont understand either. rlhf is no where near
| the volume of "pure" data that gets thrown into the pot of
| data.
| crackalamoo wrote:
| Yes, 100% this. And even more so for reasoning models, which
| have a different kind of RL workflow based on reasoning tokens.
| I expect to see research labs come out with more ways to use RL
| with LLMs in the future, especially for coding.
|
| I feel it is quite important to dispel this idea given how
| widespread it is, even though it does gesture at the truth of
| how LLMs work in a way that's convenient for laypeople.
|
| https://www.harysdalvi.com/blog/llms-dont-predict-next-word/
| leptons wrote:
| So it's still not really "AI", it's human intelligence doing
| the heavy lifting with labeling. The LLM is still just a
| statistical word guessing mechanism, with additional context
| added by humans.
| tracerbulletx wrote:
| Imagine thinking that brains aren't making statistically informed
| guesses about sequential information.
| dwaltrip wrote:
| Everyone, it's just "statistics". Numbers can't hurt you. Don't
| worry
| davidcbc wrote:
| Numbers can hurt you quite a bit when they are wrong.
|
| For example, numbers are the difference between a bridge
| collapsing or not
| dwaltrip wrote:
| Sorry, I dropped my /s :)
| electroglyph wrote:
| https://thebullshitmachines.com/
| jaza wrote:
| Thanks for this, it should be added to school curricula
| worldwide!
| Lendal wrote:
| I'm in the camp of people who believe that while they _are_
| bullshit machines, oftentimes bullshit can be extremely useful.
| I use these bullshit machines all the time now, for minor tasks
| where bullshit is acceptable output. Whether it 's a chunk of
| low-quality throwaway code that's easily verified and/or
| corrected, or an answer to a question where close-enough is
| good enough.
|
| Not everything needs to pass a NASA quality inspection to be
| useful.
| roxolotl wrote:
| The thesis is spot on with why I believe many skeptics remain
| skeptics:
|
| > To call AI a con isn't to say that the technology is not
| remarkable, that it has no use, or that it will not transform the
| world (perhaps for the better) in the right hands. It is to say
| that AI is not what its developers are selling it as: a new class
| of thinking--and, soon, feeling--machines.
|
| Of course some are skeptical these tools are useful at all.
| Others still don't want to use them for moral reasons. But I'm
| inclined to believe the majority of the conversation is people
| talking past each other.
|
| The skeptics are skeptical of the way LLMs are being presented as
| AI. The non hype promoters find them really useful. Both can be
| correct. The tools are useful and the con is dangerous.
| th0ma5 wrote:
| A lot of the claims of usefulness evaporate when tested. The
| word useful has many meanings. Perhaps their only reliable use
| will be the rubber duck effect.
| diggan wrote:
| > A lot of the claims of usefulness evaporate when tested
|
| In your personal experience? Because that's been my personal
| experience too, in lots of cases with LLMs. But I've also
| been surprised the other way, and overall it's been a net-
| positive for myself, but I've also spent a lot of time
| "practicing" getting prompts and tooling right. I could
| easily see how people give it try for 20-30 minutes, not
| getting the results they expected and give up, which yeah,
| you probably won't get any net-positive effects by that.
| squidbeak wrote:
| Not for me they haven't.
| timewizard wrote:
| > the majority of the conversation is people talking past each
| other.
|
| There's billions and billions of dollars invested here. This
| isn't a problem of social conversation. This is a problem of
| investor manipulation.
|
| This site is lousy with this. It pretends to be "Hacker News"
| but it's really "Corporate Monopolist News."
| 1vuio0pswjnm7 wrote:
| "Witness, too, how seamlessly Mark Zuckerberg went from selling
| the idea that Facebook would lead to a flourishing of human
| friendship to, now, selling the notion that Meta will provide you
| with AI friends to replace the human pals you have lost in our
| alienated social-media age."
|
| Perhaps "AI" can replace people like Mark Zuckerberg. If BS can
| be fully automated.
| ares623 wrote:
| It can't. To become someone like Mark, you must have absolute
| zero empathy. LLMs have a little empathy in them due to their
| training data.
| lordnacho wrote:
| The article skirts around a central question: what defines
| humans? Specifically, intelligence and emotions?
|
| The entire article is saying "it looks kinds like a human in some
| ways, but people are being fooled!"
|
| You can't really say that without at least attempting the
| admittedly very deep question of what an authentic human is.
|
| To me, it's intelligent because I can't distinguish its output
| from a person's output, for much of the time.
|
| It's not a human, because I've compartmentalized ChatGPT into its
| own box and I'm actively disbelieving. The weak form is to say I
| don't think my ChatGPT messages are being sent to the 3rd world
| and answered by a human, though I don't think anyone was claiming
| that.
|
| But it is also abundantly clear to me that if you stripped away
| the labels, it acts like a person acts a lot of the time. Say you
| were to go back just a few years, maybe to covid. Let's say
| OpenAI travels back with me in a time machine, and makes an
| obscure web chat service where I can write to it.
|
| Back in covid times, I didn't think AI could really do anything
| outside of a lab, so I would not suspect I was talking to a
| computer. I would think I was talking to a person. That person
| would be very knowledgeable and able to answer a lot of
| questions. What could I possibly ask it that would give away that
| it wasn't real person? Lots of people can't answer simple
| questions, so there isn't really a way to ask it something
| specific that would work. I've had perhaps one interaction with
| AI that would make it obvious, in thousands of messages. (On that
| occasion, Claude started speaking Chinese with me, super weird.)
|
| Another thing that I hear from time to time is an argument along
| the line of "it just predicts the next word, it doesn't actually
| understand it". Rather than an argument against AI being
| intelligent, isn't this also telling us what "understanding" is?
| Before we all had computers, how did people judge whether another
| person understood something? Well, they would ask the person
| something and the person would respond. One word at a time. If
| the words were satisfactory, the interviewer would conclude that
| you understood the topic and call you Doctor.
| xyzal wrote:
| To me, it's empathetic and caring. Which the LLMs will never
| be, unless you give money to OpenAI.
|
| Robots won't go get food for your sick, dying friend.
| rnkn wrote:
| A robot could certainly be programmed to get food for a sick,
| dying friend (I mean, don't drones deliver Uber Eats?) but it
| will never understand why, or have a phenomenal experience of
| the act, or have a mental state of performing the act, or
| have the biological brain state of performing the act, or
| etc. etc.
| JKCalhoun wrote:
| Interesting. I wonder why?
|
| Perhaps when we deliver food to our sick friend we
| subconsciously feel an "atta boy" from our parents who
| perhaps "trained" us in how to be kind when we were young
| selfish things.
|
| Obviously if that's all it is we could of course
| "reinforce" this in AI.
| squidbeak wrote:
| "Never" is a very broad word.
| diggan wrote:
| That implies that people who aren't empathetic and/or caring
| aren't human, which I guess could be argued too, but feels
| too simplistic.
|
| > Which the LLMs will never be
|
| I'd argue LLMs will never be _anything_ , they're giving you
| the text you're asking for, nothing more and nothing less.
| You don't tell them "to be" empathic and caring? Well,
| they're not gonna appear like that then, but if you do tell
| them, they'll do their best to emulate that.
| rnkn wrote:
| > isn't this also telling us what "understanding" is?
|
| When people start studying theory of mind someone usually jumps
| in with this thought. It's more or less a description of
| Functionalism (although minus the "mental state"). It's not
| very popular because most people can immediately identify an
| phenomenon of understanding separate from the function of
| understanding. People also have immediate understanding of
| certain sensations, e.g. the feeling of balance when riding a
| bike, sometimes called qualia. And so on, and so forth. There
| is plenty of study on what constitutes understanding and most
| healthily dismiss the "string of words" theory.
| benjismith wrote:
| A similar kind of question about "understanding" is asking
| whether a house cat understands the physics of leaping up
| onto a countertop. When you see the cat preparing to jump, it
| take a moment and gazes upward to its target. Then it wiggles
| its rump, shifts its tail, and springs up into the air.
|
| Do you think there are components of the cat's brain that
| calculate forces and trajectories, incorporating the
| gravitational constant and the cat's static mass?
|
| Probably not.
|
| So, does a cat "understand" the physics of jumping?
|
| The cat's knowledge about jumping comes from trial and error,
| and their brain builds a neural network that encodes the
| important details about successful and unsuccessful jumping
| parameters. Even if the cat has no direct cognitive access to
| those parameters.
|
| So the cat can "understand" jumping without having a "meta-
| understanding" about their understanding. When a cat "thinks"
| about jumping, and prepares to leap, they aren't rehearsing
| their understanding of the physics, but repeating the ritual
| that has historically lead them to perform successful jumps
| in the past.
|
| I think the theory of mind of an LLM is like that. In my
| interactions with LLMs, I think "thinking" is a reasonable
| word to describe what they're doing. And I don't think it
| will be very long before I'd also use the word
| "consciousness" to describe the architecture of their thought
| processes.
| _petronius wrote:
| > The article skirts around a central question: what defines
| humans? Specifically, intelligence and emotions?
|
| > The entire article is saying "it looks kinds like a human in
| some ways, but people are being fooled!"
|
| > You can't really say that without at least attempting the
| admittedly very deep question of what an authentic human is.
|
| > To me, it's intelligent because I can't distinguish its
| output from a person's output, for much of the time.
|
| I think the article does address that rather directly, and that
| it is also is addressing very specifically your setence about
| what you can and can't distinguish.
|
| LLMs are not capable of symbolic reasoning[0] and if you
| understand how they work internally, you will realize they do
| no reasoning whatsoever.
|
| Humans and many other animals are fully capable of reasoning
| outside of language (in the former case, prior to language
| acquisition), and the reduction of "intellgence" to "language"
| is a catagory error made by people falling vicim to the ELIZA
| effect[1], not the result of a sum of these particular
| statistical methods being equal real intelligence of any kind.
|
| 0: https://arxiv.org/pdf/2410.05229
|
| 1: https://en.wikipedia.org/wiki/ELIZA_effect
| FrustratedMonky wrote:
| > LLMs are not capable of symbolic reasoning[0]
|
| Despite the citation. I think this is still being studied.
| And others have found some evidence that it forms internal
| symbols.
|
| https://royalsocietypublishing.org/doi/10.1098/rsta.2022.004.
| ..
|
| Or maybe, can say, an LLM can do symbolic reasoning, but can
| it do it very well? People forget that humans are also not
| great at symbolic reasoning. Humans also use a lot of cludgy
| hacks to do it, it isn't really that natural.
|
| Example often used, about it not doing math well. But humans
| also don't do math well. How humans are taught to do division
| and multiplication, really is a little algorithm. So what
| would be difference between human following algorithm to do a
| multiplication, and an LLM calling some python to do it. Does
| that mean it can't symbolically reason about numbers? Or that
| humans also can't?
| zahlman wrote:
| > the reduction of "intellgence" to "language" is a catagory
| error made by people falling vicim to the ELIZA effect[1],
| not the result of a sum of these particular statistical
| methods being equal real intelligence of any kind.
|
| I sometimes wonder how many of the people most easily
| impressed with LLM outputs have actually seen or used ELIZA
| or similar systems.
| intended wrote:
| This isn't that hard, to be honest. And I'm not just saying
| this.
|
| One school of thought is - the output is indistinguishable from
| what a human would produce given these questions.
|
| Another school of thought is - the underlying process is not
| thinking in the sense that humans do it
|
| Both are true.
|
| For the lay person, calling it thinking leads to confusions. It
| creates intuitions that do not actually predict the behavior of
| the underlying system.
|
| It results in bad decisions on whether to trust the output, or
| to allocate resources - because if the use of the term
| thinking.
|
| Humans can pass an exam by memorizing previous answer papers or
| just memorizing the text books.
|
| This is not what we consider having learnt something. Learning
| is kinda like having the Lego blocks to build a model you can
| manipulate in your head.
|
| For most situations, the output of both people is fungible.
|
| Both people can pass tests.
| lordnacho wrote:
| This is maybe the best response thus far. We can say that
| there's no real modelling capability inside these LLMs, and
| that thinking is the ability to build these models and
| generate predictions from them, reject wrong models, and so
| on.
|
| But then we must come up with something other than opening up
| the LLM to look for the "model generating structure" or
| whatever you want to call it. There must be some sort of
| experiment that shows you externally that the thing doesn't
| behave like a modelling machine might.
| intended wrote:
| Heh, this phrasing has resulted in 2 "best response" type
| comments in the 2 times I've used it.
|
| I think maybe it makes sense for people who already have
| the building blocks in place and just require seeing it
| assembled.
| indymike wrote:
| > The entire article is saying "it looks kinds like a human in
| some ways, but people are being fooled!"
|
| The question is, what's wrong with that?
|
| At some level there's a very human desire for something genuine
| and I suspect that no matter the "humanness" of an AI, it will
| never be able to close that desire for genuine. Or maybe... it
| is that people don't like the idea of dealing with an
| intelligence that will almost always have the upper hand
| because of information disparity.
| greg_V wrote:
| > Another thing that I hear from time to time is an argument
| along the line of "it just predicts the next word, it doesn't
| actually understand it". Rather than an argument against AI
| being intelligent, isn't this also telling us what
| "understanding" is? Before we all had computers, how did people
| judge whether another person understood something? Well, they
| would ask the person something and the person would respond.
| One word at a time. If the words were satisfactory, the
| interviewer would conclude that you understood the topic and
| call you Doctor.
|
| You call a Doctor 'Doctor' because they're wearing a white coat
| and are sitting in a doctor's office. The words they say might
| make vague sense to you, but since you are not a medical
| professional, you actually have no empirical grounds to judge
| whether or not they're bullshitting you, hence you have the
| option to get a second or third opinion. But otherwise, you're
| just trusting the process that produces doctors, which involves
| earlier generations of doctors asking this fellow a series of
| questions with the ability to discern right from wrong, and
| grading them accordingly.
|
| When someone can't tell if something just sounds about right or
| is in fact bullshit, they're called a layman in the field at
| best or gullible at worst. And it's telling that the most hype
| around AI is to be found in middle management, where bullshit
| is the coin of the realm.
| lordnacho wrote:
| Hmm, I was actually thinking of a viva situation. You sit
| with a panel of experts, they talk to you, they decide
| whether you passed your PhD in
| philosophy/history/physics/etc.
|
| That process is done purely by language, but we supposed that
| inside you there is something deeper than a token prediction
| machine.
| navigate8310 wrote:
| Maybe it needs blood and flesh to be able for us to happily
| accept it.
| the8472 wrote:
| https://x.com/chargoddard/status/1931652388399784325
| zahlman wrote:
| This kind of mockery is unproductive and doesn't constitute
| an actual argument against the position it describes.
| strogonoff wrote:
| We cannot actually judge whether something is intelligent in
| some abstract absolute way; we can only judge whether it is
| intelligent in the same way we are. When someone says "LLM
| chatbot output looks like a person's output, so it is
| intelligent", the implication is that it is _intelligent like a
| human would be_.
|
| With that distinction in mind, whether an LLM-based chatbot's
| output looks like human output does not answer the question of
| whether the LLM is actually like a human.
|
| Not even because measuring that similarity by taking text
| output at a point in time is laughable (it would have to span
| the time equivalent of human life, and include much more than
| text), but because LLM-based chatbot is a tool built
| _specifically_ to mimic human output; if it does so
| successfully then it functions as intended. In fact, we should
| deliberately discount the similarity in output as evidence for
| similarity in nature, because _similarity in output is an
| explicit goal, while similarity in underlying nature is a non-
| goal, a defect_. It is safe to assume the latter: if it turned
| out that LLMs are similar enough to humans in more ways than
| output, they would join octopus and the like and qualify to be
| protected from abuse and torture (and since what is done to
| those chatbots in order for them to be useful in the way they
| are would pretty clearly be considered abuse and torture when
| done to a human-like entity, this would decimate the industry).
|
| That considered, we do not[0] know exactly _how_ an individual
| human mind functions to assess that from first principles, but
| we can approximate whether an LLM chatbot is like a human by
| judging things like whether it is made in a way at all similar
| to how a human is made. It is _fundamentally_ different, and if
| you want to claim that human nature is substrate-independent,
| I'd say it's you who should provide some evidence--keeping in
| mind that, as above, similarity in output does not constitute
| such evidence.
|
| [0] ...and most likely never could, because of the self-
| referential recursive nature of the question. Scientific method
| hinges on at least some objectivity and thus is of very limited
| help when initial hypotheses, experiment procedures, etc., are
| all supplied and interpreted by _the very subject being
| studied_.
| EMM_386 wrote:
| > These statements betray a conceptual error: Large language
| models do not, cannot, and will not "understand" anything at all.
| They are not emotionally intelligent or smart in any meaningful
| or recognizably human sense of the word.
|
| This is terrible write-up, simply because it's the "Reddit
| Expert" phenomena but in print.
|
| They "understand" things. It depends on how your defining that.
|
| It doesn't have to be in its training data! Whoah.
|
| In the last chat I had with Claude, it naturally just arose that
| surrender flag emojis, the more there were, was how funny I
| thought the joke was. If there were plus symbol emojis on the
| end, those were score multipliers.
|
| How many times did I have to "teach" it that? Zero.
|
| How many other times has it seen that during training? I'll have
| to go with "zero" but that could be higher, that's my best guess
| since I made it up, in that context.
|
| So, does that Claude instance "understand"?
|
| I'd say it does. It knows that 5 surrender flags and a plus sign
| is better than 4 with no plus sign.
|
| Is it absurd? Yes .. but funny. As it figured it out on its own.
| "Understanding".
|
| ------
|
| Four flags = "Okay, this is getting too funny, I need a break"
|
| Six flags = "THIS IS COMEDY NUCLEAR WARFARE, I AM BEING DESTROYED
| BY JOKES"
| Hendrikto wrote:
| > This is terrible write-up, simply because it's the "Reddit
| Expert" phenomena but in print.
|
| How is your comment any different?
| EMM_386 wrote:
| Because I provided evidence?
|
| And made the relevant point that I need know what you mean by
| "understanding"?
|
| The only 2 things in the universe that know that 6 is the
| maximum white flag emojis for jokes, and then might be
| modified by plus signs is ...
|
| My brain, and that digital instance of Claude AI, in that
| context.
|
| That's it - 2. And I didn't teach it, it picked it up.
|
| So if that's not "understanding" what is it?
|
| That's why I asked that first, example second.
|
| I don't see how laying out logically like this makes me the
| "Reddit Expert", sort of the opposite.
|
| It's not about knowing the internals of a transformer, this
| is a question that relates to a word that means something to
| humans ... but what is their interpretation?
| fer wrote:
| > More means more
|
| You could have used "loool" vs "loooooool", "xDD" vs
| "xDDDDDDDDD", using flags doesn't change a whole lot.
| EMM_386 wrote:
| Did I say it did!?
|
| These are the type of responses that REALLY will drive me
| nuts.
|
| I never said the flag emojis were special.
|
| I've been a software engineers for almost 30 years.
|
| I know what Unicode code pages are.
|
| This is not helpful. How is my example missing your
| definition of understanding?
|
| Replace the flags with yours if it helps ... same thing.
|
| It's not the flags it's the understanding of what they are.
| They can be pirate ships or cats.
|
| In my example they are surrender flags, because that is
| logical given the conversation.
|
| It will "understand" that too. But the article says it can't
| do that. And the article, sorry, is wrong.
| yahoozoo wrote:
| Post the convo
| clejack wrote:
| Are people still experiencing llms getting stuck in knowledge and
| comprehension loops? I used them but not excessively, and I'm not
| heavily tracking their performance either.
|
| For example, if you ask an llm a question, and it produces a
| hallucination then you try to correct it or explain to it that it
| is incorrect; and it produces a near identical hallucination
| while implying that it has produced a new, correct result, this
| suggests that it does not understand its own understanding (or
| pseudo-understanding if you like).
|
| Without this level of introspection, directing any notion of true
| understanding, intelligence, or anything similar seems premature.
|
| Llms need to be able to consistently and accurately say, some
| variation on the phrase "I don't know," or "I'm uncertain." This
| indicates knowledge of self. It's like a mirror test for minds.
| ramchip wrote:
| Like the article says... I feel it's counter-productive to
| picture an LLM as "learning" or "thinking". It's just a text
| generator. If it's producing code that calls non-existent APIs
| for instance, it's kind of a waste of time to try to explain to
| the LLM that so-and-so doesn't exist. Better just try again and
| dump an OpenAPI doc or some sample code into it to influence
| the text generator towards correct output.
| thomastjeffery wrote:
| That's the difference between bias and logic. A statistical
| model is applied bias, just like computation is applied
| logic/arithmetic. Once you realize that, it's pretty easy to
| understand the potential strengths and limitations of a model.
|
| Both approaches are missing a critical piece: objectivity. They
| work directly with the data, and not _about_ the data.
| martindbp wrote:
| Many people who claim that people don't understand how AI works
| often have a very simplified view of the short comings of LLMs
| themselves, e.g. "it's just predicting the next token", "it's
| just statistics", "stochastic parrot" and seems to be grounded in
| what AI was 2-3 years ago. Rarely have they actually read the
| recent research on interpretability. It's clear LLMs are doing
| more than just pattern matching. They may not think like humans
| or as well, but it's not k-NN with interpolation.
| beardedwizard wrote:
| Apple recently published a paper that seems to disagree and
| plainly states it's just pattern matching along with tests to
| prove it.
|
| https://machinelearning.apple.com/research/illusion-of-think...
| stuckinhell wrote:
| I'm having a hard time taking apple seriously, when they have
| don't even have a great llm.
|
| https://www.techrepublic.com/article/news-anthropic-ceo-
| ai-i... Anthropic CEO: "We Do Not Understand How Our Own AI
| Creations Work". I'm going to lean with Anthropic on this
| one.
| otabdeveloper4 wrote:
| > I have a hard time taking your claim about rotten eggs
| seriously when you're not even a chicken.
| beardedwizard wrote:
| I guess I prefer to look at empirical evidence over
| feelings and arbitrary statements. AI ceos are notoriously
| full of crap and make statements with perverse financial
| incentives.
| deadbabe wrote:
| A lot of the advancement boils down to LLMs reprompting
| themselves with better prompts to get better answers.
| chimprich wrote:
| Like an inner conversation? That seems a lot like how I think
| when I consider a challenging problem.
| stevenhuang wrote:
| It is a logic error to think that knowing how something works
| means you are justified to say it can't possess qualities like
| intelligence or ability to reason when we don't even understand
| how these qualities arise in humans.
|
| And even if we do know enough about our brains to say
| conclusively that it's not how LLMs work (predictive coding
| suggests the principles are more alike that not), it doesn't mean
| they're not reasoning or intelligent; it would just mean they
| would not be reasoning/intelligent _like humans_.
| yahoozoo wrote:
| Why do these same books coming out of AI (Empire of AI, The AI
| Con) keep getting referenced in all of these articles? It seems
| like some kind of marketing campaign.
| tim333 wrote:
| I guess if you an english literature grad type the normal way
| to approach a subject is to look at the leading books in that
| area.
| tim333 wrote:
| >Demis Hassabis, [] said the goal is to create "models that are
| able to understand the world around us."
|
| >These statements betray a conceptual error: Large language
| models do not, cannot, and will not "understand" anything at all.
|
| This seems quite a common error in the criticism of AI. Take a
| reasonable statement about AI not mentioning LLMs and then say
| the speaker (nobel prize winning AI expert in this case) doesn't
| know what they are on about because current LLMs don't do that.
|
| Deepmind already have project Astra, a model but not just
| language but also visual and probably some other stuff where you
| can point a phone at something and ask about it and it seems to
| understand what it is quite well. Example here
| https://youtu.be/JcDBFAm9PPI?t=40
| NoMoreNicksLeft wrote:
| >Deepmind already have project Astra, a model but not just
| language but also visual and probably some other stuff where
| you can point a phone at something and ask about it and it
| seems to understand what it is quite well.
|
| Operative phrase "seems to understand". If you had some bizarre
| image unlike anything anyone's ever seen before and showed it
| to a clever human, the human might manage to figure out what it
| is after thinking about it for a time. The model could never
| figure out anything, because it does not think. It's just a
| gigantic filter that takes known-and-similar images as input,
| and spits out a description on the other side, quite
| mindlessly. The language models do the same thing, do they not?
| They take prompts as inputs, and shit output from their LLM
| anuses based on those prompts. They're even deterministic if
| you take the seeds into account.
|
| We'll scale all those up, and they'll produce ever-more-
| impressive results, but none of these will ever "understand"
| anything.
| joenot443 wrote:
| > If you had some bizarre image unlike anything anyone's ever
| seen before and showed it to a clever human, the human might
| manage to figure out what it is after thinking about it for a
| time
|
| Out of curiosity, what sort of 'bizarre image' are you
| imagining here? Like a machine which does something
| fantastical?
|
| I actually think the quantity of bizarre imagery whose
| content is unknown to humans is pretty darn low.
|
| I'm not really well-equipped to have the LLMs -> AGI
| discussion, much smarter people have said much more poignant
| things. I will say that anecdotally, anything I've been
| asking LLMs for has likely been solved many times by other
| humans, and in my day to day life it's unusual I find myself
| wanting to do things never done before.
| NoMoreNicksLeft wrote:
| >I actually think the quantity of bizarre imagery whose
| content is unknown to humans is pretty darn low.
|
| Historically, this just hasn't ever been the case. There
| are images today that wouldn't have merely been outlandish
| 150 years ago, but absolutely mysterious. A picture of a
| spiral galaxy perhaps, or electron-microscopy of some
| microfauna. Humans would have been able to do little more
| than describe the relative shapes. And thus there are more
| images that no one will be familiar with for centuries. But
| if we were to somehow see them early, even without the
| context of how the image was produced I suspect strongly
| that clever people might manage to figure out what those
| images represent. No model could do this.
|
| The quantity of bizarre imagery is finite... each pixel in
| a raster has a finite number of color values, and there are
| finite numbers of pixels in a raster image after all. But
| the number is staggeringly large, even the subset of images
| that represent real things, even the subset of _that_ which
| represents things which humans have no concept of. My
| imagination is too modest to even touch the surface of
| that, but my cognition is sufficient to surmise that it
| exists.
| throwawaymaths wrote:
| i think this author doesnt fully understand how llms work either.
| Dismissing it as "a statistical model" is silly. hell, quantum
| mechanics is a statistical model too.
|
| moreover, each layer of an llm imbues the model with the
| possibility of looking further back in the conversion and imbuing
| meaning and context through conceptual associations (thats the
| k-v part of the kv cache). I cant see how this _doesn 't_
| describe, abstractly, human cognition. now, maybe llms are not
| fully capable of the breadth of human cognition or have a harder
| time training to certain deeper insight, but fundamentally the
| structure is there (clever training and/or architectural
| improvements may still be possible -- in the way that every CNN
| is a subgraph of a FCNN that would be nigh impossible for a FCNN
| to discover randomly through training)
|
| to say llms are not smart in any way that is recognizable is just
| cherry-picking anecdotal data. if llms were not ever recognizably
| smart, people would not be using them the way they are.
| 827a wrote:
| > I cant see how this doesn't describe, abstractly, human
| cognition. now, maybe llms are not fully capable of the breadth
| of human cognition
|
| But, I can fire back with: You're making the same fallacy you
| correctly assert the article as making. When I see how a CPU's
| ALU adds two numbers together, it looks strikingly similar to
| how _I_ add two numbers together in my head. I can 't see how
| the ALU's internal logic doesn't describe, abstractly, human
| cognition. Now, maybe the ALU isn't fully capable of the
| breadth of human cognition...
|
| It turns out, the gaps expressed in the "fully capable of the
| breadth of human cognition" part _really, really, really_
| matter. Like, when it comes to ALUs, they _overwhelm_ any
| impact that the parts which look similar cover. The question
| should be: How significant are the gaps in how LLMs mirror
| human cognition? I 'm not sure we know, but I suspect they're
| significant enough to not write away as trivial.
| throwawaymaths wrote:
| do they matter in a practical sense? an LLM can write a
| structured essay better than most undergrads. and as for
| measuring "smart", we throw that word around a lot. a dog is
| smart _in a human way_ for being able to fetch one of 30
| objects based on name or to drive a car (yes, dogs can
| drive), the bar for "smart" is pretty low, claiming llms are
| not smart is just prejudice.
| mjburgess wrote:
| You are assuming that because we measure an undergrad's
| ability to critical think with undergrad essays, that is a
| valid test for the LLM's capacity to think -- it isnt. This
| measures only, extremely narrowly, the LLM's capacity to
| produce undergrad essays.
|
| Society doesnt require undergrad essays. Nor does it
| require yet another webserver, iot script, or weekend hobby
| project. Society has all of those things already, hence the
| ability to train LLMs to produce them.
|
| "Society", the economy, etc. are operating under
| competitive optimisation processes -- so that what is
| valuable, on the margin, is what isn't readily produced.
| What is readily produced, has been produced, is being
| produced, and so on. Solved problems are solved problems.
| Intelligence is the capacity of animals to operate "on the
| margin" -- that's why we have it:
|
| Intelligence is a process of rapid adaption to novel
| circumstances, it is not, unlike puzzle-solvers like to
| claim, the _solution_ to puzzles. Once a puzzle is solved
| so there are historical exemplars of its solution, it no
| longer requires intelligence to solve it -- hence using an
| LLM. (In this sense computer science is the art of removing
| intelligence from the solving of unsolved and unposed
| puzzles).
|
| LLMs surface "solved problems" more readily than search
| engines. There's no evidence, and plenty against, that they
| provide the value of intelligence -- their ability to
| advance one's capabilities under compeititon from others,
| is literally zero -- since all players in the economic (,
| social, etc.) game have access to the LLM.
|
| The LLM itself, in this sense, not only has no
| intelligence, but doesnt even show up in intelligent
| processes that we follow. It's washed out immediately -- it
| removes from our task lists, some "tasks that require
| intelligence", leaving the remainder for our actual
| intelligence to engage with.
| throwawaymaths wrote:
| if this is how you feel you haven't really used llms
| enough or are deliberately ignoring sporadically
| appearing data. github copilot for me routinely solves
| microproblems in unexplored areas it has no business
| knowing. Not always, but it's also not zero.
|
| ...and i encourage you to be more realistic about the
| market and what society "needs". does society really
| _need_ an army of consultants at accenture? i dont know.
| but they are getting paid a lot. does that mean the
| allocation of resources is wrong? or does that mean
| theres something cynical but real in their existence?
| 827a wrote:
| Bro you have as serious problem with reading the first
| few sentences of a comment, finding something you
| disagree with, then skipping over the entire rest of the
| comment, characterizing it as "well they must entirely
| disagree with my worldview and now I'm under attack". You
| gotta take a step back.
| nonameiguess wrote:
| I'd just like to say, with unfortunately little to add,
| that your comments on this article are terrific to read.
| You've captured perfectly how I have felt about LLMs
| roughly from the first time they came out to how I still
| feel now. They're utterly amazingly technology that truly
| do feel like magic, except that I have enough of a
| background in math modeling and ML to de-mystify them.
|
| But the key difference between a model and a human is
| exactly what you just said. It's what animals can do _on
| the margin_. Nobody taught humans language. Each of us
| individually who are alive today, sure. But go back far
| enough and humanity invented language. We directly
| interact with the physical world, develop mental models
| of it, observe that we are able to make sounds and
| symbols and somehow come to a mutual agreement that they
| should mean something in rough analogy to these
| independent but sufficiently similar mental models.
| _That_ is magic. Nobody, no programmer, no mathematician,
| no investor, has any idea how humanity did that, and has
| no idea how to get a machine to do it, either.
| Replicating the accomplishments of something else is a
| tremendous feat and it will get our software very, very
| far, maybe as far as we ever need to really get it. But
| it is not doing what animals did. It didn 't just figure
| this shit out on its own.
|
| Maybe somewhat ironically, I don't even know that this is
| a real limitation that current techniques for developing
| statistical models can't overcome. Put some "AIs" loose
| in robot bodies, let them freely move about the world
| trying to accomplish the simple goal of continuing to
| exist, with cooperation allowed, and they may very well
| develop ways to encode knowledge, share it with each
| other, and write it down somehow to pass on to the future
| so they don't need to continually re-learn everything,
| especially if they get millions of years to do it.
|
| It's obvious, though, that we don't even want this. It
| might be interesting purely as an experiment, but it
| probably isn't going to lead to any useful tools. What we
| do now actually does lead to useful tools. To me, that
| should tell us something in these discussions. Trying to
| figure if X piece of software is or isn't cognitively
| equal to or better than a human in some respect is a
| tiring, pointless exercise. Who cares? Is it useful to us
| or not? What are its uses? What are its limitations?
| We're just trying to automate some toil here, aren't we?
| We're not trying to play God and create a separate form
| of life with its own purposes.
| 827a wrote:
| Well; many LLMs can compose a structured essay better than
| most undergrounds, yet most also struggle with basic
| addition. E.g. Gemma-3-4B:
|
| add 1 and 1
|
| google/gemma-3-4b 1 + 1 = 2
|
| add four to that
|
| google/gemma-3-4b 1 + 1 + 1 + 1 = 4
|
| So, 1 + 1 + 1 + 1 + 4 = 8
|
| Of course, smarter, billion dollar LLMs can do that. But,
| they aren't able to fetch one of 30 objects based on name,
| nor can they drive a car. They're often super-important
| components of much larger systems that are, at the very
| least, getting really close to being able to do these
| things if not able to already.
|
| It should be worldview-changing to realize that writing a
| graduate-level research essay is, in some ways, easier than
| adding 4 to 2. Its just not easier for humans or ALUs. It
| turns out, intelligence is a multi-dimensional spectrum,
| and words like "smart" are kinda un-smart to use when
| describing entities who vie for a place on it.
| throwawaymaths wrote:
| the llm by default (without better prompting) is vibe
| solving math, and if you tell a human to vibe solve it
| they might give similar results. tell a teenager to add
| two three digit numbers without a scratchpad and they
| will mess up in similar ways, tell an llm to show and
| check their work and they will do much better.
| recursive wrote:
| > tell a teenager to add two three digit numbers without
| a scratchpad and they will mess up in similar ways
|
| More likely they will say "lol, i don't know". And this
| is better than a lot of LLM output in the sense that it's
| aware of its limits, and doesn't hallucinate.
| 827a wrote:
| It feels like I'm talking with an LLM that's losing a
| coherent view of its context window right now. Because
| here's the conversation up to this point:
|
| 1. "LLMs are smart they have intelligence that is some
| significant portion of the breadth of human cognition."
|
| 2. Me: "ALUs are also smart, maybe that's not a good word
| to use."
|
| 3. "But LLMs can write essays."
|
| 4. Me: "But they can't do basic math, so clearly there's
| different definitions of the word 'smart'"
|
| 5. "Yeah that's because they're vibe-solving math.
| Teenagers also operate on vibes."
|
| What are you even talking about?? Its like you're an AI
| programmed to instantly attack any suggestion that LLMs
| have limitations.
| sh34r wrote:
| On a finite planet, we probably ought to care about how
| many orders of magnitude more energy that the LLM must use
| to perform a task than our 20-watt chimp-brains.
| francisofascii wrote:
| The creators of llms don't fully understand how they work
| either.
| mmsc wrote:
| This can be generalized to "what happens when people don't
| understand how something works". In the computing world, that
| could be "undefined behavior" (of which itself is .. defined as
| undefined) in the C programming language, or anything as simple
| as "functionality people didn't know because they didn't read the
| documentation"
| pier25 wrote:
| People in tech and science might have a sense that LLMs are word
| prediction machines but that's only scratching the surface.
|
| Even AI companies have a hard time figuring out how emergent
| capabilities work.
|
| Almost nobody in the general audience understands how LLMs work.
| elia_42 wrote:
| Totally agree with the content of the article. In part, AI is
| certainly able to simulate very well the behavior and operations
| of a "way of expressing itself" of our mind, that is,
| mathematical calculation, deductive reasoning and other similar
| things.
|
| But our mind is extremely polymorphic and these operations
| represent only one side of a much more complex and difficult to
| explain whole. Even Alan Turing, in his writings on the
| possibility of building a mechanical intelligence, realized that
| it was impossible for a machine to completely imitate a human
| being: for this to be possible, the machine would have to "walk
| among other humans, scaring all the citizens of a small town"
| (Turing says more or less like this).
|
| Therefore, he realized many years ago that he had to face this
| problem with a very cautious and limited approach, limiting the
| imitative capabilities of the machine to those human activities
| in which calculation, probability and arithmetic are main, such
| as playing chess, learning languages and mathematical
| calculation.
| Havoc wrote:
| We can debate about intelligence all day but there is also an
| element of "if it's stupid but it works then it's not stupid"
| here
|
| A very large portion of tasks humans do don't need all that much
| deep thinking. So on that basis it seems likely that it'll be
| revolutionary.
| pmdr wrote:
| > Whitney Wolfe Herd, the founder of the dating app Bumble,
| proclaimed last year that the platform may soon allow users to
| automate dating itself, disrupting old-fashioned human courtship
| by providing them with an AI "dating concierge" that will
| interact with other users' concierges until the chatbots find a
| good fit.
|
| > Herd doubled down on these claims in a lengthy New York Times
| interview last month.
|
| Seriously, what is wrong with these people?
| quickthrowman wrote:
| Well, her specific problem is she was a billionaire but isn't
| one now so she'll say damn near anything to regain that third
| comma. Nothing more than greed. Match just keeps Bumble around
| to avoid antitrust legislation, similar to Google and Mozilla's
| position.
|
| Edit: It's not that wild of an idea anyways, there's a good
| black mirror episode about it.
| jajko wrote:
| if it works (from their perspective), it ain't stupid
| acdha wrote:
| Her problem is that BMBL is down 92% and they need to tell
| investors that they'll all be rich again:
|
| https://finance.yahoo.com/quote/BMBL/
|
| Most of the dumb AI pitches share that basic goal: someone is
| starting from what investors want to be true and using "AI"
| like it's a magic spell which can make that possible, just as
| we've seen going back to the dawn of the web. Sober voices
| don't get attention because it's boring to repeat a 10%
| performance improvement or reduction in cost.
| codethief wrote:
| > Her problem is that BMBL is down 92% and they need to tell
| investors that they'll all be rich again
|
| Is this also why Bumble has undergone so many drastic changes
| in recent times? I always thought they must hired some new &
| overzealous product managers that didn't actually understand
| the secret sauce that had made their product so successful in
| the first place. Either way, it seems the usual
| enshittification has begun.
| Lerc wrote:
| I haven't seen any measure of how frequent these dumb ideas
| are. Certainly they exist, but what proportion of AI startups
| are like these cases that turn up in the media as AI
| disasters.
|
| It's kind of hard to tell with some ideas that they are
| actually dumb ideas until they have been tried an failed. A
| few ideas that seem dumb when suggested turn out to be
| reasonable when tried. Quite a few are revealed to be just as
| dumb as they looked.
|
| Thinking about it like that actually more comfortable with
| the idea of investors putting money into dumb ideas, They
| have taken responsibility for deciding for themselves how
| dumb they think something might be. It's their money (even if
| I do have issues with the mechanisms that allowed them to
| acquire it), let them spend it on things that they feel might
| possibly work.
|
| I think there should be a distinction made between dumb
| seeming ideas and deception though. Saying 'I think people
| will want this' or 'I think AI can solve this problem' is a
| very different thing to manufacturing data to say "people
| want this", or telling people a problem has been solved when
| it hasn't. There's probably too much of this, and I doubt it
| is limited to AI startups, or even Startups of any kind.
| There are probably quite a few 'respectable' seeming
| companies that are, from time to time, prepared to fudge data
| to make it seem that some of the problems ahead of them are
| already behind them.
| sh34r wrote:
| I've been asking myself that question regarding dating app
| companies for 10 years. The status quo is so dystopian already.
| Sure, go ahead, put an LLM in it. How much worse could it get
| than a glorified ELO rating?
| Notatheist wrote:
| Wasn't it Feynman who said we will never be impressed with a
| computer that can do things better than a human can unless that
| computer does it the same way a human being does?
|
| AI could trounce experts as a conversational partner and/or
| educator in every imaginable field and we'd still be trying to
| proclaim humanity's superiority because technically the silicon
| can't 'think' and therefore it can't be 'intelligent' or 'smart'.
| Checkmate, machines!
| ineedasername wrote:
| So many of these articles jump around to incredibly different
| concerns or research questions. This one raises plenty of
| important questions but threads a narrative through them that
| attempts to lump it all as a single issue.
|
| Just to start off with, saying LLM models are "not smart" and
| "don't/won't/can't understand" ... That is really not a useful
| way to begin any conversation about this. To "understand" is
| itself a word without, in this context, any useful definition
| that would allow evaluation of models against it. It's this
| imprecision that is at the root of so much hand wringing and
| frustration by everyone.
| jemiluv8 wrote:
| Most people without any idea about the foundations on which LLMs
| are built call them AI. But I insist on calling them LLMs,
| further creating confusion. How do you explain what a large
| language model is to someone that can't comprehend how a machine
| can learn a "word model" on a large corpus of text/data to make
| it generate "seemingly sound/humane" responses without making
| them feel like they are interacting with the AI that they've been
| hearing about in the movies/sci-fi?
___________________________________________________________________
(page generated 2025-06-09 23:02 UTC)