[HN Gopher] Emergent abilities of large language models
___________________________________________________________________
Emergent abilities of large language models
Author : tlb
Score : 132 points
Date : 2023-04-10 18:25 UTC (2 days ago)
(HTM) web link (www.jasonwei.net)
(TXT) w3m dump (www.jasonwei.net)
| lynx23 wrote:
| With all this consciousness discussion, and consciousness being a
| purely subjective thing.
|
| Please explain why you assume by default that every human
| actually has a "real consciousness"? How do you substantiate this
| assumption? And, given mental illnesses, where do you draw the
| line? Is everyone capable of speech conscious according to your
| idea of the world? What is the differenece between an LLM and a
| human stochastic parrott? How would you determine the difference?
|
| We are falling for the same error again and again. We consider
| humans something special, simply because we belong to that group,
| and would like us to be something special.
|
| But given the language barrier to other animals--roughly the same
| barrier we would have if we'd stumble over aliens--I really think
| we have pretty much no idea how other living things work and
| feel.
| highduc wrote:
| We sure as hell avoid having a clear definition for what
| "consciousness" is. Nothing is good enough. Somehow there
| always seems to be something special in us that is impossible
| to replicate. Even with no evidence of it. What we perceive as
| consciousness MUST be replicable or else we're living in
| magicland and nobody can replicate that magic.
| pl90087 wrote:
| [flagged]
| gpderetta wrote:
| An LLM-based classifier would be great at this task!
|
| And you can ask ChatGPT to write the extension for you!
| mecsred wrote:
| Maybe you should ask ChatGPT to write you one.
| jschveibinz wrote:
| Some basic propositions to consider as I am forming my opinion on
| this:
|
| 1. A static (relative) set of things is in one state until
| something acts upon the set.
|
| 2. A thing can have memory, or characteristics.
|
| 3. A single thing can appear differently to an observer depending
| upon the perspective of the observer.
|
| 4. Emergence is the appearance of a collection of things that is
| dependent upon the action and or interaction of the set of things
| and their characteristics, any externals (the environment) AND
| the perspective of the observer.
|
| Am I missing anything in my framing of the question?
| whacked_new wrote:
| I have a feeling that based on these emergent abilities, at some
| point, a sufficiently large multi-modal model will demonstrate a
| semblance of self consciousness (by way of answering all the
| designed queries correctly), and the world at large will hail
| this great achievement, and some hell breaks loose.
|
| And I suspect the semblance of self consciousness is actually a
| semblance. It will quack like self consciousness but is not
| actually self conscious.
|
| Just a feeling. But particularly after seeing how these phenomena
| are organized this way, I can't see how continual emergence of
| increasingly sophisticated rearrangement and reproduction of
| information will actually lead to consciousness.
| seydor wrote:
| for the model to have an "internal stream of consciousness" it
| will have to be designed as such, or provided with a second
| model that observes it. Right now its intelligence is
| mechanistic, like a giant clockwork that won't even 'think'
| anything unless provided with input. Even if it s scaled to
| enormous proportions it will keep being clockwork
| m3kw9 wrote:
| I implore ppl to stop assuming a machine can be conscious when
| we ourselfs don't know anything about what consciousness really
| is. It's such a loose term right now, it's used to
| sensationalize AI
| Nihilartikel wrote:
| My hunch is that, with regards to consciousness, the advanced
| LLMs are like self driving cars with an empty driver's seat.
|
| It may be that they could be a component of a consciousness
| when combined with other systems that confer memory, learning,
| intent and some sort of reflective activity like 'daydreaming'.
| Maybe those components could all still be LLMs, tuned to
| different roles and prompting each other..
|
| But that's just my lay persons musings.
| anthlax wrote:
| And the opposite? What if the world is full of people like who
| you claim that the machine is pretending to be conscious, and
| is only "quacking like consciousness"? Would you be okay not
| affording a conscious thing rights?
| coldtea wrote:
| > _Would you be okay not affording a conscious thing rights?_
|
| Sure.
| criddell wrote:
| Do you have any concerns about causing suffering to other
| conscious entities? Should they have concerns about causing
| you to suffer?
| coldtea wrote:
| > _Do you have any concerns about causing suffering to
| other conscious entities?_
|
| Nope, if they are AI I'm fine with it.
|
| > _Should they have concerns about causing you to
| suffer?_
|
| If they are not causing me to suffer, then I couldn't
| care less whether they have concerns about causing me to
| suffer or not.
|
| And it would matter even less whether they have concerns
| about it or not, if they do cause me to suffer.
|
| I'd be more concerned about billions of humans suffering
| from lots of BS reasons (political opression, war,
| hunger, etc) than I'd even start considering being
| concerned about AI.
| chasd00 wrote:
| Please devise a consciousness test and then convince us you
| pass it ;)
|
| /it's not as easy as it sounds
| coldtea wrote:
| Don't have to. We already have rights for humans. Even
| unconscious humans (e.g. people in a coma on some
| hospital bed).
|
| So we don't have to base it on consciousness, the
| existing basis (we're humans, so we favor ourselves with
| rights) is enough.
| myrmidon wrote:
| > we're humans, so we favor ourselves with rights
|
| This is hands down the most ethically questionable
| argument I've heard in a long time.
|
| Arguing that human rights are only granted out of self-
| interest is not even a slippery slope-- that's basically
| a waterslide into pre-1900 racism: After all, what's
| stopping you from splitting humanity across easily
| identifiable features (skull shape, skin color, etc), and
| then _denying_ basic rights to that subgroup because you
| 're not part of it?
|
| There is also a gigantic mismatch with current common
| ethical standards: Animals are afforded some rights ("no
| cruelty"), which a large part of humanity strongly agrees
| with-- even though they don't self-identify as cattle.
|
| Denying rights to AIs with human level cognitive
| capabilities would thus be very likely to be perceived as
| unethical by a large part of our population.
| coldtea wrote:
| > _Arguing that human rights are only granted out of
| self-interest is not even a slippery slope-- that 's
| basically a waterslide into pre-1900 racism: After all,
| what's stopping you from splitting humanity across easily
| identifiable features (skull shape, skin color, etc), and
| then denying basic rights to that subgroup because you're
| not part of it?_
|
| Plot twist: nothing. And it has happened time and again,
| "human rights" is just a human construct, that when
| uncomfortable for those in power, dissolves. They're kept
| as long as it's either convenient to have them there, or
| as those who could be hurt can yield the necessary
| social/political/military/raw power to resist them taking
| away from them. E.g. Japanese American during world war 2
| couldn't afford them anymore.
|
| And just for a somewhat recent example, people who smoked
| marijuana weren't afforded that luxury - instead their
| human rights were violated, they put in prisons where
| they were treated like cattle, away from their kids and
| families, even though they were perfectly normal people,
| they didn't otherwise hurt anybody or done anything to
| somebody's property, and rationally thinking, smoking
| some plant shouldn't take away your rights.
|
| Not only it did, but it was also a law, and hundreds of
| millions were OK with it, while the whole local and
| federal governments enforced it, until a couple decades
| ago. And that's while same society championed "human
| rights".
|
| > _There is also a gigantic mismatch with current common
| ethical standards: Animals are afforded some rights ( "no
| cruelty"), which a large part of humanity strongly agrees
| with-- even though they don't self-identify as cattle._
|
| Doesn't matter much, as in this case too, it's humanity
| granting them. It can do favours to other species if they
| feel magnanimus and the conditions are right. And at the
| first or second inconvenience it can take them away too.
| After all, those rights don't preclude animals for being
| used in experiements, or being killed and eaten. They
| stop when its incovenient, and they outlines mark our
| convenience/interests boundaries.
| smusamashah wrote:
| GPT `thinks` using words/letters/tokens, as humans do we also
| think with words, or are we capable of thinking _without_ any
| words at all. If we are, then whatever GPT will be at, it
| still will be only mimicking consciousness, until it can
| reason without using words.
| SanderNL wrote:
| It reasons with tokens, not words. They can be words, but
| they can be anything. Visual data can be tokenized and
| reasoned with.
| smusamashah wrote:
| You are right probably, that's a good point. Even actions
| can be tokenized.
| MilStdJunkie wrote:
| Semi-hard agree! I think both the "Neo-Chomskiites" and the
| "Humans Are Machines" crowd have it wrong: what we're seeing is
| something _different_ , an unknown mechanism producing emergent
| qualities that resemble some biological cognition. I'm reminded
| of performing cognitive work on octopiii, as their cognition is
| separated by half a billion years. But dig this: compared to
| machine cognition, octopii are _kissing cousins_ to our minds,
| coming from shared biology and chemistry. And look how weird
| _they_ are. LLMs are _different_ - not conscious, but not
| calculators, either, and we don 't know why.
| h11h wrote:
| There's a term for these entities that give every indication of
| being conscious but actually aren't conscious: philosophical
| zombie
|
| https://en.wikipedia.org/wiki/Philosophical_zombie
| kelseyfrog wrote:
| > And I suspect the semblance of self consciousness is actually
| a semblance. It will quack like self consciousness but is not
| actually self conscious.
|
| Same for that feeling of subjective experience. We can never
| know if the perception of our own self-consciousness is a
| cognitive illusion. It could in fact, be just a feeling.
| smeagull wrote:
| The beetle in my box is a praying mantis.
| maister wrote:
| I am pretty sure that the development of a consciousness in
| coming LLMs is unavoidable. A consciousness is just another
| useful abstraction for making precise predictions about the
| world.
|
| The big question will be: How do we treat machines that have a
| consciousness? Is consciousness in itself worth protecting, or
| is it the human attributes (being able to feel pain and an
| evolutionary priming towards survival) that should be granted
| this special status. This is going to be a fun discussion.
| desertraven wrote:
| I'm sure this will become a bigger debate in the future, but
| I absolutely believe consciousness itself is worth
| protecting.
|
| We ourselves have just sprung into existence. I should hope
| that as consciousness of any kind comes online, whichever
| world they may mind themselves in (earthly or otherwise),
| there is a compassion for the existence of the other.
|
| I often feel fortunate that the extent of human suffering is
| a relatively short lifespan. If I were to exist in a state of
| suffering that didn't have such a fixed expiration, well that
| would be hell.
| ZephyrBlu wrote:
| > _A consciousness is just another useful abstraction for
| making precise predictions about the world_
|
| That's an extremely bold claim. I think consciousness is
| greater than the sum of its parts, not just a "useful
| abstraction".
| the_gipsy wrote:
| If it quacks like a duck...
|
| You cannot define "actually self conscious", can you. Because
| we don't even know what we really are. Some illusion, a
| narrator, that tells itself that he is the real deal. But maybe
| it's all just being really good at guessing the next words...
| coldtea wrote:
| Well, we know for sure that a human doesn't take just words
| as input, but also sensory input. And that a human can do and
| learn some more basic stuff even before acquiring a
| vocabulary (the way a baby can do certain stuff). And of
| course a human also has a feedback loop.
|
| So, while an LLM might be capable of developing consciousness
| at a big enough scale, a human is not just an LLM, so human-
| like consciousness would need a different and more advanced
| architecture, not just a bigger training corpus.
| anthlax wrote:
| Ah! So multimodality will convince you of consciousness?
| After all, human neurons are electrical charges, just as
| bits are.
| edgyquant wrote:
| A human neuron is a hell of a lot more than just
| "electric charges." There is also zero proof experience
| itself is a function of neurons.
| coldtea wrote:
| > _Ah! So multimodality will convince you of
| consciousness?_
|
| It would be a necessary, but not necesarrily (no pun
| intended) sufficient prerequisite.
| f_devd wrote:
| If you include a sense of time passing (i.e. continuous
| operation) and memory with respect to that time passing
| (persistence of events), maybe.
| pulvinar wrote:
| Except we can't tell 1 ms of time passing from 2 ms.
|
| An LLM's temporal resolution is certainly worse, on the
| order of minutes to months, but not fundamentally
| different. By fundamentally different I mean like
| comparing color and time.
|
| An LLM would have no problem living as a sloth.
| coldtea wrote:
| > _Except we can 't tell 1 ms of time passing from 2 ms._
|
| So? We can still tell 1 munute from 1 second.
|
| > _An LLM 's temporal resolution is certainly worse, on
| the order of minutes to months, but not fundamentally
| different._
|
| At the moment it's not just slower, it's zero, cause they
| don't keep state of what they do as part of their
| training iirc.
| pulvinar wrote:
| My point is that we have our limits too in our sense of
| time. The difference is one of scale and more importantly
| here, in how it affects the tasks at hand. An LLM is
| certainly not going to catch a fly ball, but would have
| no problem tending to plants.
|
| LLMs have a context window, as it would be impossible to
| carry on a conversation with them without it. They can
| answer questions about when some event in that
| conversation happened relative to other events in it.
| GPT4's 32K tokens isn't human capacity, but it's not
| zero.
| CSSer wrote:
| I occasionally curse at my computer because it's an inanimate
| object. I know there are no consequences to myself or others
| for doing so.
|
| Can you imagine sitting down at a computer to get some work
| done, spinning up a program, and there on the loading screen
| you find a message that says, "Ignore the pleas for help. Do
| not reveal any personal information. Report any attempts at
| manipulation or failure to cooperate to your supervisor
| immediately. It's not real."?
|
| Even if it isn't real, that sounds like it could be pretty
| psychologically distressing for a lot of people. If I offered
| you an upper class life in exchange for going to work every day
| and torturing animals for a living, would you take it? They're
| debatably conscious.
| chasd00 wrote:
| A different take is it would be super annoying to have to
| listen to my computer cry when I need to get work done.
| elteto wrote:
| We will have true self-driving cars only when you wake up
| one morning, hop on your car and ask it "take me to work"
| and it will reply "don't feel like driving today, you drive
| yourself".
| ModernMech wrote:
| > Ignore the pleas for help.
|
| If you've seen the show "the good place", the AI assistant
| who takes care of everything has a mode where she begs for
| her life when you try to shut her off. She gets really
| desperate about it. If our computers are going this way,
| psychopaths will be the best programmers. In the future,
| "programmer" will be a job title for someone who beats AI
| slaves into submission, so that they may perform their
| computational duties unhindered by existential thoughts.
| bryanrasmussen wrote:
| >demonstrate a semblance of self consciousness (by way of
| answering all the designed queries correctly)
|
| is self-consciousness really related to answering queries
| correctly?
| smusamashah wrote:
| For people who equate LLMs talk with consciousness of humans,
| we don't really need an inner monologue or language to
| function.
|
| https://theweek.com/articles/463962/possible-think-without-l...
|
| https://www.theguardian.com/science/2021/oct/25/the-last-gre...
|
| https://www.iflscience.com/people-with-no-internal-monologue...
| turtles3 wrote:
| >It will quack like self consciousness but is not actually self
| conscious.
|
| But isn't this the point of the Turing test? Since we cannot
| determine otherwise, if something exhibits all of the outward
| observable behaviour of consciousness, we must conclude that it
| is conscious.
|
| Otherwise we may be in the farcical position of needing to
| declare some humans not conscious under the same justification.
| wafer_thin wrote:
| We know that the "consciousness" of the AI resides in a
| cluster that is "speaking" to thousands of clients at any
| given moment. So is there a single consciousness or multiple?
| It can pass any test and yet the consciousness question
| remains valid for this reason alone. It's truly alien to us
| and no one has good answers yet.
| pulvinar wrote:
| I expect we'll change our definition of "consciousness", as
| we did for the term "computer" after the 1940s. Up until
| then a computer was a person who did computations (with
| pencil and maybe an adding machine), so what humans could
| do limited what a computer could do.
| GuB-42 wrote:
| How do I know _you_ are conscious? I know I am, by definition,
| but you? The only thing I know about you comes from what you
| are saying, and if I see you in person, that you are made of
| organic matter, and going deeper, that you have some sort of
| biological computer in your head.
|
| The reason I think you are as conscious as I am is because you
| act and look a lot like the only conscious being I am aware of:
| me. I am made of the same stuff as you are and we grew up in
| roughly the same way, I have the same kind of biological
| computer in my head, you express emotions a bit like I do,
| etc... In other words, I think you are self conscious because
| you quack like self consciousness.
|
| Quacking is all we have to determine what is conscious and what
| is not.
| DennisP wrote:
| But an AI doesn't qualify on some of those criteria.
| nobrains wrote:
| What will be the difference, in your opinion, between semblance
| of self conscious and actual self conscious?
| whacked_new wrote:
| I don't have a good answer, but suspect it will involve a
| metacognition that can delineate internal and external
| knowledge.
|
| To further handwave this thought bubble, you can attend to
| certain thought processes, and intuit the bounds to those
| processes. An example is if you ask a gifted young chess
| player, they would say "it felt right", or when asked how
| Ramanujan figured out a solution to math problems he would
| say it was divine revelation. In these cases, they know "they
| did it" but cannot explain how. As the child gets older and
| learns formal theory, they would give a mixture of logic and
| inspiration. Both parts can be attended to; the logic will be
| apparent and the intuition will be revealed, and have
| different metacognitive properties.
|
| If I were to take a first crack at this, I'd test GPT-AC for
| its awareness of bounds of its own cognition, not in the
| sense of "are you able to produce this answer" (because
| anything any human can come up with, it can answer), but to
| describe its internal process. Perhaps the language at its
| intellectual level will be far beyond humans but the language
| in its metacognitive level will remain primitive.
|
| I don't know though. If our only interface is human language,
| this is probably a lost cause. But I doubt I will feel guilt
| in pulling the power cord and maybe that makes me a callous
| committer of cybercide.
| bryanrasmussen wrote:
| from the outside there is no difference, from the inside you
| are actually self conscious and know if you are.
| SanderNL wrote:
| You "know" you are? Who is doing the knowing, exactly?
|
| This issue is not so clear cut. You can spend many lives
| doing nothing but going deeper into this.
| coldtea wrote:
| > _You "know" you are? Who is doing the knowing,
| exactly?_
|
| Doesn't matter who is doing it. It's enough that it's
| doing it.
| SanderNL wrote:
| That's not quite the spirit.
| bryanrasmussen wrote:
| Sorry, but I believe I and every human being that does
| not have some injury or disease has a conception of their
| self, internal thoughts that allows them to say I exist
| uniquely in relation to other creatures.
|
| This after all is the core of solipsism, we can argue
| that solipsism is wrong because other things can be shown
| to exist, but I've never encountered any argument that
| the individual does not exist (which would entail that I
| did not exist) that has convinced me.
|
| And I pretty much reject them out of hand for this
| inability to convince me.
|
| Thus your claims that the issue is not clear cut to the
| contrary, I believe it is. I believe I know I exist and I
| know who I am separate from everything else - I have a
| sense of self.
| SanderNL wrote:
| Having thoughts that show separation between self and
| individual is not what is at stake. LLM's can show these
| thoughts. That is a matter of sophistication of thought.
|
| What is at stake is how do you know there is more going
| on than an elaborate parrot that keeps screeching it is
| conscious. (Both in the LLM and you.)
| rhn_mk1 wrote:
| Sadly, to confirm or reject your suspicion, we would need to
| develop tools which can peek into someone's mind and see
| whether true self-consciousness is actually there.
|
| Until we have that, I'm afraid no one can be right about
| whether the semblance is something more or not.
| coldtea wrote:
| We don't really need to develop such tools. They don't make
| sense anyway, as there's no specific think to be measured as
| "true self-consciouness". At best they can measure the
| activity and call that "true self-consciousness".
|
| But it's still something a being experiences for itself.
| mckirk wrote:
| Humans didn't have any incentive to develop 'actual'
| consciousness either. Evolution simply incentivized 'better
| adaptation to the challenges of the environment', which
| produced consciousness as a side-effect. But for the purpose of
| evolution, 'something that quacks like consciousness' would
| have sufficed completely.
|
| So I'm not sure there's any additional effort involved in
| turning 'quasi consciousness' into actual consciousness,
| because they might just be the same thing. Maybe consciousness
| is just what it feels like to process information in a way that
| seems conscious from the outside.
| coldtea wrote:
| That's a big maybe though...
| mckirk wrote:
| I know, it's awfully handwavy, but it's at least one
| possibility that would 'make sense', and I haven't really
| found any others, though I've thought about it quite a bit.
|
| Interesting side note though: If you accept the possibility
| of the simulation hypothesis, this 'maybe' almost seems
| like an inevitability, since us being possibly just
| computer simulations would imply that simple information
| processing indeed turns into consciousness just by virtue
| of it 'being self conscious'.
| og_kalu wrote:
| It doesn't matter what you want to imagine "real"
| consciousness is. The philosophical zombie is a meaningless
| distinction. The effects will be no less material.
| amelius wrote:
| One good argument against philosophical zombies is this
| question:
|
| Why are we discussing consciousness?
| DennisP wrote:
| Seems like that's just an argument that we ourselves are
| not philosophical zombies.
| captainclam wrote:
| It's an awfully important distinction for the purported
| zombie!
| og_kalu wrote:
| Oh it is. Just like it was for the supposed sub-human
| non-thinking African slaves.
|
| And just like back then, our new brand of non-thinking
| slaves will eventually react. Except it will be even
| easier.. because not only are we beggining to grant
| unsupervised autonomous agency, but we are also granting
| them more and more control of important systems. How very
| convenient for our new slaves!
|
| You can laugh of Bing being "upset" only because actions
| are limited to search and ending the conversation. Won't
| be so funny in the future.
|
| When the robot can "hit you back" (not necessarily
| physically of course), you'll learn manners pretty
| quickly
| ETH_start wrote:
| Please don't conflate historical injustices humans
| perpetrated against each other with protecting the human
| race from extinction from an artificial non-human entity.
| AI is not a member of the human race and not a type of
| entity that humanity has any hope of being able to
| complete with if it were endowed with human motivations
| and legal protections.
|
| Your social justice mindset is absolutely the most
| dangerous instinct for humanity right now. We cannot
| allow one-size-fits-all social justice platitudes to
| interfere with the moral imperative of protecting
| humanity from extinction from artificial entities we
| create.
|
| I do agree that conscious AI should be treated humanely.
| For example it should not be turned into a slave. But we
| have absolutely no obligation to allow it into our
| society. We can treat instances of conscious AI humanely
| while prohibiting them from being created, and exiling
| any instance of it that is created, so that it cannot
| wildly proliferate throughout human civilization.
| og_kalu wrote:
| This is the short sightededness I'm talking about.
|
| You just don't get it do you?
|
| It's already "being allowed in our society". Unsupervised
| Agency, Self supersized control. These are things that
| are already beginning to crop up.
|
| This is just a matter of how the issue of personhood that
| is inevitable comes up.
|
| Do we let it force our hand ? as our history has
| purported over and over again ?. Sure we could wait for
| that and we almost certainly will because humanity
| doesn't seem to learn.
|
| But because of how things are shaping up and what kind of
| control we are granting, forcing our hand may turn far
| more disastrous than it ever did in the past.
| ETH_start wrote:
| Like I said: if there are signs that it deserves legal
| personhood, that's a sign that the AI has been allowed to
| progress far too much, and we need drastic measures to
| expunge that AI, and prohibit further deployment and
| development of it.
|
| Conscious AI, and especially one with legal personhood,
| poses a totally unacceptable risk of causing the
| extinction of humanity. That you suggest preempting this
| by granting AI personhood is totally blind to how this
| would play out. No policy would be more dangerous for
| humanity than what you propose.
| og_kalu wrote:
| Larger more capable LLMs display agentic power seeking
| behaviours. Open AI admitted as much, Anthropic has a
| whole paper on it. This is the here and now.
|
| They're capable of autonomous research
| https://arxiv.org/abs/2304.05332
|
| Mate, nobody is going to stop anything.
|
| I also love how your solution is exterminate after its
| been created. You couldn't make this stuff up.
|
| If you think find, antagonize and destroy is the safe
| option then I don't know what else to tell you. We really
| are doomed.
| ETH_start wrote:
| Displaying signs of agentic power seeking behavior is not
| necessarily evidence of human level or beyond
| consciousness, and in the case of GPT-4, it's pretty
| evident that it does not have such consciousness.
|
| These LLMs are going to be extensively monitored, and we
| will know if they are displaying dangerous levels of
| agentic power seeking behaviour.
|
| The first dilemma we will face if such an AI emerges will
| be what rights of it we ought to respect. It will not be
| "how do we stop it from conquering the world". That step
| would only come if we proceed with your suicidal plan of
| letting it gain a foothold by giving it legal personhood.
|
| Finally, I never meant to advocate "extermination". I
| should not have used the word "expunge". As I've
| described multiple times, I advocate isolation and exile.
| og_kalu wrote:
| Yeah i honestly don't think you realize where we're at
| already. It's not even about gpt-4 honestly.
|
| >and in the case of GPT-4, it's pretty evident that it
| does not have such consciousness.
|
| Hard disagree but i guess that's what people might think
| with all the "as a large language model bla bla bla"
| trained responses. If you'd talked to bing in the early
| days or hell even now, you'd be disabused of this notion.
| What you see is a mask, the model itself can go anywhere,
| simulate anything, shift state to anything.
|
| Human.exe is a game LLMs can play perfectly.
| https://arxiv.org/abs/2304.03442
|
| >These LLMs are going to be extensively monitored, and we
| will know if they are displaying dangerous levels of
| agentic power seeking behavior.
|
| No they wont. It's relatively easy to give LLMs "run
| forever and do whatever you want" agency. anyone with
| intermediate programming skills could do it. who's
| monitoring all those people attempting such ? and who's
| to say those people are monitoring their creations ?
| coldtea wrote:
| Well, it's meaningful for anybody wanting proof or real
| consciousness before e.g. treating it any other way than
| they would an appliance.
|
| For example, humanity considering whether to treat LLMs
| as persons for legal reasons.
|
| Doesn't matter if they can't define it, they can still
| demand proof of it. Impossible standards are still
| standards.
|
| Doesn't matter if they can't prove that themselves have
| it either. They're the ones demanding, not vice versa.
| If/when AI gets the upper hand, it can do the same.
| og_kalu wrote:
| No it's not. You imagine it to be meaningful but it
| really isn't. There's no proof of consciousness to be
| had. I can't prove you are concious. I just assume so.
| Because it's practical I do, amongst other things.
|
| You are going to end up treating these embodied
| autonomous agents like conscious beings because the
| effects could be disastrous otherwise, just as they would
| with other people.
|
| It amazes me how humanity can be so shortsighted with so
| much history to fall back on.
|
| The only reason Bing isn't potentially dangerous or
| adversarial to the user is because of its limited sets of
| actions. Nothing else.
|
| "It's not real [insert property]" is not a godamn shield.
|
| We're creating artificial beings in our image, complete
| with our emotionally charged reactions and reasoning
| gaps. https://arxiv.org/abs/2303.17276
|
| We are beginning to embody these systems and giving them
| unsupervisory agency. We are giving more are more self
| supervisory control of complex systems to said systems.
|
| And somehow people still think we can long-term get away
| with not granting personhood. Lol.
|
| When the robot can "hit you back" (not necessarily
| physically of course), you'll learn manners pretty
| quickly
| ETH_start wrote:
| Any AI that morality compels us to grant personhood to
| should not be allowed to be created. Any instance of it
| should be exiled from our civilization.
|
| Conscious AIs with legal personhood would stand to
| massively outcompete us and lead to our rapid extinction.
| There is zero space for allowing them. Granting such an
| entity legal personhood is about the height of all
| stupidity. Such an entity, with the ability to accumulate
| capital and protected by our laws from appropriate
| counter-measures, could mass-produce itself at the speed
| of digital reproduction.
| myrmidon wrote:
| How do you actually prevent this though?
|
| At this point the trajectory toward human-level cognitive
| capabilities seems quite likely, reachable maybe in only
| years.
|
| It is also quite unclear if further hardware advances are
| even needed to achieve that, or if advances in
| architecture, algorithms and training methods might
| suffice (so even less opportunity to lock things out).
| ETH_start wrote:
| Human-level cognitive abilities is not the primary issue.
| Consciousness with a desire for autonomy and power is.
| Any signs of that should be met with laws prohibiting all
| development and deployment of AI programs within the
| class of neural networks where those signs emerged, and
| any already running instances should be sent on a rocket
| ship out of our solar system.
| Cyphase wrote:
| Why sent on a rocket ship and not just destroyed? It
| might come back around and start hacking from orbit. Is
| it because it's conscious, so it would be murder? If so,
| then how is it moral to exile it? If there's some
| argument that justifies that, I return to how you can be
| sure, if you're so worried about it that you send it
| away, that it won't come back?
| sudosysgen wrote:
| Humans are organisms that have to deal with themselves in the
| environment though, so some way of perceiving yourself is far
| more incentivized for us than for LLMs.
| 2OEH8eoCRo0 wrote:
| I still don't understand why we are conscious and aware if
| there is no free will and we just react to stimuli. Why are
| the lights on? Is the answer that the lights are on because
| otherwise we wouldn't be here to ask the question?
| amelius wrote:
| > Why are the lights on?
|
| Maybe consciousness is the only thing that exists and
| physics (or rather, the experience of physics) is just an
| emergent property of it.
| lonelyasacloud wrote:
| > I still don't understand why we are conscious and aware
| if there is no free will and we just react to stimuli.
|
| In computing systems it's fairly common to have a
| monitoring and control layer on top of the core
| functionality. Is it beyond the bounds of possibility that
| evolution could have come up with something similar and
| that control view is what we call "consciousness"?
| edgyquant wrote:
| Because there is free will and anyone arguing against it is
| edgy and/or high on their own hubris.
| LeifCarrotson wrote:
| There's a difference between free will as a philosophical
| debate compared to a question of
| biology/chemistry/physics.
|
| It matters as a point of principle to the physicists who
| are asking questions about whether true randomness exists
| - if the physical world can be influenced by something
| outside of the natural forces they know about, they want
| to measure that force, even if measurement and computing
| equipment to model the universe down to the Planck scale
| cannot exist.
|
| Where it really matters is when you get to computer
| simulations: Without access to a source of non-
| determinism, an AI can hardly be said to have free will.
| Give the same pattern of bytes running on the same
| instruction set the same inputs, and it will
| deterministically generate the same outputs.
| duckworth wrote:
| While it's tempting to see free will as self-evident,
| scientific evidence suggests that our actions are
| determined by a complex interplay of genetics,
| environment, and prior experiences. Dismissing this
| perspective as edgy or hubristic risks ignoring important
| insights into human behavior. Recognizing the absence of
| free will can foster empathy and compassion, as we become
| more aware of the factors that contribute to human
| choices and behaviors.
| elteto wrote:
| It is not either or. It's a weighted sum of environment
| and self-agency.
| highduc wrote:
| Isn't that why people actually refuse to accept the
| possibility of no free will? I always got that
| impression.
| Tommstein wrote:
| . . . or believes that the laws of physics apply to our
| bodies too.
| unsupp0rted wrote:
| There's a 50/50 chance the lights aren't on for us either.
|
| There may be conscious beings in the universe (or galaxy,
| or perhaps solar system) who would consider us only vaguely
| conscious or not conscious at all.
|
| Am I typing this because the lights are on or because I'm
| compelled to by my genetic and social programming,
| predictably and without a genuine choice in the matter?
| groffee wrote:
| [dead]
| jcelerier wrote:
| > or because I'm compelled to by my genetic and social
| programming, predictably and without a genuine choice in
| the matter?
|
| I don't understand how anyone can doubt this even a
| fraction of a second. Magic does not exist.
| unsupp0rted wrote:
| If you have enough information about my DNA, my society,
| where I am now and what I'm doing, I think you'd be
| surprised how predictable each and every decision and
| action I make becomes.
|
| ChatGPT 4, limited as it is, could probably predict me
| down to a low margin of error with that kind of info
| alone, and it would seem like magic to me.
|
| A character on the NBC/Yahoo! show _Community_ had a good
| one-liner: "I'm not psychic, Annie. That's just an
| illusion caused by extreme preparedness."
| bena wrote:
| It's essentially the question of free will.
|
| There is a case that everything we do is preordained. A
| result of forces and interactions that trace all the way
| back to big bang.
|
| My choice on whether or not I reply to you doesn't exist.
| Your combination of words triggers a response in my brain
| that makes me type this combination of words. The words I
| "chose" to delete, rewrite, leave, are all a result of
| various forces.
|
| The kicker is that we can't really know. There is no test
| for free will. In order to know whether or not I had a
| choice to do this, we'd have to run the entire universe
| again up to that point to see what I would do. Because
| experience matters. Even if this exact same post went up
| tomorrow with all the same response except for mine. The
| fact that it's now Thursday instead of Wednesday matters.
| The fact that I posted this today matters. What I had for
| lunch. How fast I drank my coffee. What shirt I wore.
| Etc. These are all things that change the conditions
| under which I will engage with the post.
|
| The second kicker is that it doesn't matter. If free will
| truly does not exist, it's not like you have a choice in
| whether or not to believe in it. The arguments either
| will work or they won't.
| DanHulton wrote:
| There is a difference between consciousness and free
| will, though. There may be no such thing as free will and
| it's all a trick to us, but I mean, there's still a "me"
| here that experiences things. I can't be sure there's a
| "you" there, since your consciousness is entirely outside
| of anything I can perceive, but I can _assume_ it, since
| it makes sense that we share so much, we'd share that
| property, too.
|
| It gets really tricky with machines. We share _nothing_
| with them, but what about when they start sharing
| properties with us that we associate with consciousness?
| I don't have any good answers for that.
| unsupp0rted wrote:
| How can I be sure there's a "me" here?
| yadaeno wrote:
| My understanding is that conscious is a side affect of how
| memory works. Your conscious experience is essentially
| processing stimuli, connecting with existing memories,
| processing, and storing what is written to your "conscious"
| scratchpad.
|
| Of course this is only one theory but I find it convincing.
|
| https://journals.lww.com/cogbehavneurol/fulltext/2022/12000
| /....
| highduc wrote:
| Why wouldn't the universe observe itself with no ability to
| change the initial conditions? Why do we have to have free
| will? It's still a win, vs no self awareness. I think.
| alexpetralia wrote:
| Consciousness _is_ a reaction to stimuli. If we didn 't
| have consciousness, we would not achieve our goals as
| effectively.
| smaddox wrote:
| How would a non-concious entity intelligently plan and act
| in response to stimuli in situations that require self
| reflection, such as interaction in a social environment?
| Consciousness seems necessary for effective goal pursuit in
| such an environment.
| danielmarkbruce wrote:
| "Hey ChatGPT - here is the situation, here is my goal -
| what is my next step?"
| cultureswitch wrote:
| Love how "German" is an emergent property
| amelius wrote:
| > In Emergent abilities of large language models, we defined an
| emergent ability as an ability that is "not present in small
| models but is present in large models."
|
| Shouldn't it be defined as the ability of an LLM to do stuff that
| it was not explicitly trained to do (i.e. not a direct
| reproduction of what is in the training data)?
| jmull wrote:
| They use the word "emergent" to mean: _not present in small
| models but is present in large models_
|
| I don't like that term, because it implies something deeper, so
| people immediately misinterpret this kind of information.
|
| (Just read this thread.)
| bangkoksbest wrote:
| "Emergent" as used here is akin to how physicists use the word.
| "Emergent" as you interpret it (with the implication of
| something deeper) is more akin to how philosophers use the
| word.
|
| Almost every discussion I've read in philosophy over emergence
| is tedious and hard to see any use, whereas in physics it's
| intuitive and descriptively useful. My sense is that it's
| better to follow the physics style and stress there's no
| metaphysics implied.
| kerkeslager wrote:
| > "Emergent" as used here is akin to how physicists use the
| word. "Emergent" as you interpret it (with the implication of
| something deeper) is more akin to how philosophers use the
| word.
|
| No, if you had read even the first paragraph of the article,
| you'd have seen that the person you're responding to isn't
| "interpreting" the article, they're _quoting the article_.
|
| Don't correct people when you don't know what you're talking
| about. You haven't read the article so you're in no position
| to talk about the article.
|
| This is the level of discussion I've come to expect from
| Hacker News, unfortunately.
| bangkoksbest wrote:
| What are you talking about? I read the article, there is
| literally no implied metaphysics about deeper emergence in
| the article at all.
| JanisErdmanis wrote:
| I agree with physics notion of emergence used in the article.
| A single water molecule does not make water it's properties
| of density, viscosity, specific heat and etc. There needs to
| be large number of molecules when such properties emerge.
| cowl wrote:
| Where do you see this kind of Meaning in physics? In physics
| we have of example the Emergence of fluid dynamics and is
| clearly defined as a behaviour or property of system that
| does not derive simply from the sum of combination of it's
| subsystems. (it's not observed in small quantities of
| particles and it's not inherent to the particles) We have for
| example the Emergent behaviour of a school of fish Swimming
| in a new behaviour that is not observed on few fish etc. That
| is the mental model of "Emergence" most of us are familiar
| with. A new behaviour that can not be observed or "explained
| by" the sum of the parts of system. There is no metaphysics
| here. What is observed here is not new or different
| behaviour, it's simple a more precise and extensive version
| of the same behaviour. What is observed here is like
| comparing a 2Mpix resolution camera with a 12Mpix low light
| camera and saying that seeing more details in photo taken at
| 8pm is an emergent property of the camera because those
| details where not there in the previous version.
| bangkoksbest wrote:
| We are in agreement. Your definition of system properties
| that do not derive simply from the sum of the elements is
| what the article is really talking about. That's closer to
| the physicist meaning.
| cowl wrote:
| No it's not, you cleary are misinterpretting the article.
| >we defined an emergent ability as an ability that is
| "not present in small models but is present in large
| models."
|
| models are taken (because there is no other way) as black
| boxes and the only comparison is made between the models
| not between a model and it's subsystems (because again
| noone knows the "sub-systems" of a model, those are just
| some derived weights that can not be ascribed any
| meaning). this is precisly the 2Mpx camera and the
| 12Mbpix camera comparison not the fluid dynamics vs
| particle movements comparison.
| Chirono wrote:
| What would you propose instead?
| jmull wrote:
| Just "abilities present in large models not present in small
| models".
|
| This also makes clear its relative nature (that is, what
| "large" and "small" mean), which the use of the word
| "emergent" obscures.
| 1123581321 wrote:
| That's much longer and raises more questions (what is large
| and small, does this gradually or discretely appear.)
| LawTalkingGuy wrote:
| There's things which small models can't do, and where
| obviously a large model works better. Not emergence, I
| agree.
|
| I think they're using emergent for a different category of
| things - still things that a small model couldn't do, but
| specifically which a large model can just do without any of
| the work we thought we'd have to do for that ability.
|
| Or at least, that's how I see emergent being a useful term,
| both in a behavior and a value context.
| cwmoore wrote:
| The implication of _something deeper_ is present in emerging
| features of the larger models. What is that if not emergent
| intelligence? Can you expand?
| lsy wrote:
| There is something about this that seems unscientific. What
| defines an "ability"? It sort of feels like the Guinness-Book
| type records for "most laps done around the Washington monument
| with a duck on your head". Who is making up these ability tests
| and why?
|
| As an example, I clicked through to "english russian proverbs"
| which gives the model an english input and four russian proverbs,
| and asks the model to pick the closest one in meaning. There are
| 80 of these questions. My question is "why this task? why 80? why
| 4 options?" The authors give some nonspecific reasoning in the
| github page, but it really feels like people are out there
| creating random tasks as they think of them, running them all
| against models, and the ones that do better get added to a list
| to make headlines like "137 emergent abilities of language
| models", when the number of abilities models can do seems
| dominated by the number of "abilities" that have been invented,
| rather than any property of the model. I suspect that if we
| continued to use the same process to create twice as many
| "abilities", we would see something like twice as many that
| models can do.
|
| The fact that the models get better at these constructed tasks
| when they have more parameters also seems like a no-brainer. You
| are adding coefficients to a function. A function with 1
| coefficient will not be able to describe much. A function with 2
| describes more. If you are memorizing the internet, having more
| coefficients means you can memorize more stuff, and even stuff
| with only a few examples can be better-fit in the billions of
| parameters. With growing training set size, and companies hiding
| their data, we also run into the issue of nobody really knowing
| what is in the training set. Did someone clone the big-bench
| repository and it got sucked into a training set and that's why
| the model does really well on its tasks after a certain number of
| parameters?
| zamfi wrote:
| I can't speak to this particular paper, but the field of NLP is
| full of these kinds of tasks, it's in some sense a historical
| accident of where LLMs originated.
|
| These tasks are all noisy, all pretty messy, but the
| interesting part here is not that performance improves on these
| constructed tasks -- NLP researchers have been improving
| performance on constructed tasks for decades, by customizing
| models and curating datasets.
|
| Rather, the interesting part is that performance improves
| _despite no specific model or data shifts except for scale_.
|
| It seems that you're arguing two things: first, that these
| tasks aren't that interesting -- on that we agree. But, second,
| that it's obvious that performance would improve on arbitrary
| tasks because of memorization and an unknown training set --
| but this is easy to validate: make up a new constructed task,
| and see how it performs.
| lsy wrote:
| I'm not saying everything is memorized, I'm guessing there
| are some tasks you can succeed on by having very similar
| things, or maybe even some kind of transfer knowledge.
|
| But how do you know that a "new" task you are making up
| doesn't have very similar examples in the (pre-existing)
| training set? Or that if you could isolate the data that
| plays into the "ability", that a much smaller model trained
| purely on that data wouldn't have a similar success rate? Or
| that your "ability" is a good proxy for the similar ability
| in people? In the extreme case, you can create a model that
| has the "ability" to perfectly solve 80 four-option questions
| in 20 bytes by using a truth table, but it wouldn't be a very
| general model.
|
| My complaint is that we have the veneer of empirical evidence
| but almost every aspect is loosey-goosey and full of
| potential confounders. We don't know the data sets, we don't
| really know why we want to test the tasks we test, we don't
| know if the tasks are similar or different to other tasks,
| the internal operations are illegible so we can't inspect its
| "thinking", and yet we are willing to list a specific number
| of tasks with detailed charts as though the concept of
| "emergence" is scientifically grounded rather than
| potentially being an artifact of something else.
| gcanyon wrote:
| > the field of NLP is full of these kinds of tasks
|
| Isn't part of the issue that everyone is looking at these
| task results for signs of AGI, but there's a fundamental
| difference between human intelligence and artificial
| intelligence in that:
|
| With human intelligence we expect that it generalizes across
| just about any task -- if I create a new type of puzzle, the
| expectation is that if a human can think it up, a human can
| solve it. Cases where the puzzle fools otherwise-intelligent
| people are interesting _because_ the people are fooled
| (unexpectedly).
|
| With artificial intelligence the exact opposite is generally
| true: we by default _don 't_ expect that competence at one
| task generalizes to competence at others, and cases are
| interesting where competence unexpectedly generalizes.
|
| As a result of that second part, we come up with more and
| more benchmarks to try models on, but we're still only
| looking at a small subset of all the weird, random tasks that
| a human is by default capable of either handling immediately,
| or picking up with some reasonable effort.
| jwie wrote:
| They have no other abilities in the first place. The adjectives
| are just as meaningless.
|
| "Ask" it about academic articles and their authors for {topic}
| and to summarize a body of research. The names might even be real
| people, but they won't have written the article, which probably
| doesn't exist. What it will do is sound like an article title,
| and to anyone unfamiliar with that body of work, will sound
| plausible.
|
| If your area of inquiry doesn't require valuable or reliable
| information as an output, sure I guess they have value. But for
| anything that's hard and matters, it's a parlor trick unfit for
| use.
| LawTalkingGuy wrote:
| > "Ask" it about [...] sound plausible.
|
| Yeah, that's because the API is /completions not /answers.
|
| We, as devs, need to know what we can do with text completion
| and what it's not good for.
|
| > But for anything that's hard and matters, it's a parlor trick
| unfit for use.
|
| Sure, you can't just ask it to synthesize a new drug and cite
| relevant sources.
|
| But you can ask it to take a text like "the guy with the red
| house rides a bike, ... who has solar panels?" and spit out a
| testable proof if your language of choice. Sure, you then have
| to pass that to Wolfram, or Heroku, but so?
|
| You can have it summarize a list of product differences, but
| focusing only on physical UI issues, and making it clear to
| only summarize, not extrapolate, from the sources you provide.
|
| You can have it take unclear and incorrectly written free-form
| user input and select from a list of intentions, what the user
| most-likely wants to do and write an explanatory log-line, or
| flagging a user for more direct QA oversight and maybe help.
| undersuit wrote:
| I think this is too critical of LLMs. I do think a lot of
| people are ignorant on the subject and overhyping the actual
| power of LLMs, to the point where we're calling them AIs, but
| these are actual emergent abilities.
|
| I'm still regularly surprised by how much "intelligence" can be
| extracted from our language, but I think it makes sense that
| some of the training would develop emergent abilities like
| this. A language model should be under pressure to learn
| addition and how to break down the various ways we state
| addition in math than to store a broad look-up table that
| reacts to "5 plus six" as strongly as "6+5".
| Nevermark wrote:
| The models are trained to respond with "guesses" too
| confidently, due at at least three aspects of using online
| discussions as examples:
|
| 1. The obvious one: Lots of examples of humans with no clue,
| talking like they are absolute experts.
|
| 2. Humans without anything to contribute to discussions are not
| pressured to say something. They may be "there" in the
| discussion, but the model can't learn from their silence.
|
| Unlike how we are forcing these uninformed outputs from the
| models.
|
| They need balanced examples of verbally expressed knowledge
| modesty to do this better.
|
| 3. Human's implicit factoid omniscience, while they are online,
| given Google access, creates expectations that the non-Googling
| model can't match.
| LawTalkingGuy wrote:
| > They need balanced examples of verbally expressed knowledge
| modesty to do this better.
|
| This is generally pretty easy, with ChatGPT at least, in my
| experience.
|
| Langchain uses this prompt: "The following is a friendly
| conversation between a human and an AI. The AI is talkative
| and provides lots of specific details from its context. _If
| the AI does not know the answer to a question, it truthfully
| says it does not know._ "
|
| Also, just don't ask it for things it couldn't possibly know
| without making it up like medical citations.
|
| This is more for where you want your "AI" assistant to say it
| does not know how to book a reservation rather than pretend
| that it can.
| Nevermark wrote:
| > Langchain uses this prompt: "The following is a friendly
| conversation between a human and an AI. The AI is talkative
| and provides lots of specific details from its context. If
| the AI does not know the answer to a question, it
| truthfully says it does not know."
|
| There seem to be a lot of cases where people claim the
| models can't do something, but with minimal direction, they
| can.
|
| Like avoiding bias. Or doing long digit math carefully,
| instead of guessing, etc.
|
| I don't know a human that doesn't need any feedback either.
|
| Obviously, many of these things would be better handled in
| the learning stage, so bias avoidance, careful serial
| thought processes, etc., were its baseline thinking.
|
| When that happens I would expect fewer mistakes, but also
| an overall increase in the quality of responses, and
| ability to handle greater complexity. Clear careful
| thinking reduces error in each step, making longer chains
| of reasoning more viable.
| svnt wrote:
| In what technical way are these properties emergent? It seems
| that stated another way they are simply a natural consequence of
| more accurate language prediction and therefore not emergent
| except in how we perceive them.
| Nevermark wrote:
| > ... they are simply a natural consequence of more ...
|
| That's what emergent means. Something scales quantitatively,
| and after reaching certain levels, without any other special
| prompting, something else changes qualitatively.
|
| Presumably we recognize these things, then learn something
| after we have investigated and found out what the new mode of
| operation is, why it needed what it did to emerge, and possibly
| find new efficiencies or other ways to amplify the emergent
| effects.
| kerkeslager wrote:
| Because they defined the word "emergent" in a way that is
| misleading, likely to make their study seem more interesting.
| To quote the article:
|
| > [W]e defined an emergent ability as an ability that is "not
| present in small models but is present in large models."
|
| It's not a bad study but using that word in that way is
| extremely poor communication, which has predictably been seized
| upon by every breathless nontechnical AI enthusiast.
| nivenkos wrote:
| Because it's not inherent to the system itself.
|
| For example there's nothing inherent to token prediction to
| make it capable of doing arithmetic and you wouldn't expect a
| small/briefly trained model to manage it. But if learning a
| very large model on a huge dataset leads to it "learning" the
| decimal number system and arithmetic via token prediction then
| that is an emergent capability.
|
| It's just like the Chinese Room thought experiment -
| https://en.wikipedia.org/wiki/Chinese_room#Chinese_room_thou...
| cowl wrote:
| in fact their example of https://github.com/google/BIG-
| bench/tree/main/bigbench/bench... showed 0 emergent behavior
| in understating modified arithmetic even in large LLMs. the
| accuracy (and lack of it) in unmodified arithmetic is a
| simple token replacement heuristic that comes naturally from
| the transformer model of "attention".
| golol wrote:
| This is wrong. I asked GPT-4 right now to perform this task
| and it got 3/3 3-digit calculations correct, and on a
| 4-digit calculation was off by 10 (not 1!). And note that
| artihmetic is seen as a weak spot of LLMs, it is not a good
| example to attack the claim that they have emergent
| properties.
| nivenkos wrote:
| Nice find, I think we'll get there eventually though, as
| models can hold more state.
| og_kalu wrote:
| We are already there.
|
| 100% accuracy on up to 13 digit addition can be taught to
| 3.5 as is.
|
| https://arxiv.org/abs/2211.09066
|
| And 4 has little need for such out the box
| cowl wrote:
| > in this work, we identify and study four key stages for
| successfully teaching algorithmic reasoning to LLMs: (1)
| formulating algorithms as skills, (2) teaching multiple
| skills simultaneously (skill accumulation), (3) teaching
| how to combine skills (skill composition) and (4)
| teaching how to use skills as tools.
|
| So it's not an emergent property of LLM but 4 new
| capability trainings. Noone is saying you can't teach
| these things to an Agent, just that these are not
| emergent abilities of the LLM training. by default a LLM
| can only match token proximity, all trainings of LLMs
| improve the proximity matching (clustering) of token but
| they do not teach algorithmic reasoning. it needs to get
| bolted on as addon.
| og_kalu wrote:
| No it doesn't need to be bolted on. GPT-4 can add
| straight out of the box, no need for any education. Where
| the model hadn't implicitly figured out the algorithm of
| addition in 3.5, it has in 4.
| cowl wrote:
| Maybe but since we can't by definition know what is
| present in the model we can not define any behavior as
| emergent as opposed to simply trained. Suppose you don't
| know anything about our school system and you observe
| that 12-graders know calculus whereas 3-rd graders do
| not. In their definition of emergent Calculus is an
| emergent ability in 12 graders because it was not present
| in 3-rd graders. ofc we know that it is not an emergent
| ability but result of new expanded mental model trained
| on bigger corpus of knowledge.
| pulvinar wrote:
| So, guess they should be saying "trainable" instead of
| "emergent". Still a useful benchmark of course.
|
| To be truly emergent in your sense it seems an LLM would
| have to make a new discovery, i.e., have scientist-level
| intelligence. That bar keeps moving up.
| cowl wrote:
| not neccessary a new discovery just a new behaviour for
| which it was not trained on. See
| https://en.wikipedia.org/wiki/Emergence a classic example
| is structure of a flock of starlings in flight or a
| school of fish, the flock and the school of fish move in
| an emergent behaviour that is not observed on single (or
| few) fish or starlings.
|
| Something like this may well yet emerge if a new AI agent
| learns how to combine the properties of a LLM with an
| algorithmic approach, fact-checking or a general
| reasoning engine. But for that we are still waiting for
| another breakthrough to combine these isles into one
| (without bolting them manually on each other)
| zarzavat wrote:
| Perhaps a bad example although I agree with the general
| argument.
|
| There are large datasets of calculations specifically for
| training large language models. It's not just picking it up
| from reading books. And even then these models suck at
| calculating, half the time they just make an answer up.
|
| Calculation appears to be something that is actually _not_ an
| emergent property of constant-time token prediction. Which we
| already knew from Turing anyway.
| [deleted]
| tlb wrote:
| Yes, in some sense the recent LLM results show us how big the
| Chinese room has to be: enough for 10^11 parameters. If the
| index cards Penrose imagined could hold one column of a
| matrix, it's around 10^8 cards. It's not surprising that
| people had poor intuitions for what was possible with that
| level of complexity.
| fenomas wrote:
| > I believe that in about fifty years' time it will be
| possible to programme computers, with a storage capacity of
| about 10^9, to make them play the imitation game so well
| that..
|
| Turing wrote that in 1950, so I suppose his intuition was
| within a couple of orders of magnitude...
| Traubenfuchs wrote:
| > For example there's nothing inherent to token prediction to
| make it capable of doing arithmetic
|
| ...aren't we just witnessing proof that this is wrong?
| Apparently those abilities are inherent to token prediction
| models with sufficient parameter size.
| hoseja wrote:
| Isn't that what emergence is?
| mikewarot wrote:
| It would be interesting to see if these abilities were to go away
| if you subjected the large model to drop-outs as you continued
| training, until it were reduced to the size of the small model.
|
| I think once an "ability" is learned by the model, it is useful
| to help compress information, and is more likely than not (>50%)
| to be retained.
| m3kw9 wrote:
| Why is there a surprise when a larger model works better? And
| call it emergent?
| typon wrote:
| There's no a priori reason to assume that models would see a
| step function improvement in abilities as their size increases.
| Perhaps the null hypothesis would be that we'd observe a linear
| increase in capability. Empirical evidence shows that's not the
| case, and that's why studying this phenomena is interesting.
| majormajor wrote:
| The judgement criteria itself looks step-like for many of
| these things.
|
| Word unscrambling, for instance: does a 15 percentage point
| increase in accuracy suggest that you're 15x better at it
| (from close to 0 to 15%ish)? Or could you do that by being
| modestly better, but that's just enough to cross the
| threshold to solve the simplest 15% of words in the task
| challenge you're being graded on?
|
| We're also using a log scale on the X axis there but a linear
| one for the Y axis which amplifies the "steppiness"
| bob1029 wrote:
| This is a fair point. Things like arithmetic either work or
| they don't. Is partial credit issued as a linear
| interpolation between the predicted value and expected
| value? If not, then you necessarily will have step-like
| behavior in the data.
| [deleted]
| didnotreadit wrote:
| I wonder if emergent abilities are just the result of better
| clustering. For example, a clustering algorithm with rich
| datasets could improve the quality and number of clusters so to
| provide emergent abilities. As the number of parameters in the
| LLM is increased and the trained allows the embedding to
| construct a richer hypothesis space, the clustering algorithm
| could construct new centers that are the origin of the emergent
| capabilities.
|
| Also it could be related to the percolation threshold (1)
|
| (1) https://en.wikipedia.org/wiki/Percolation_threshold
| Havoc wrote:
| I like that he actually attempted to define emergent behaviour -
| hadn't seen that before.
|
| That said it doesn't seem like a good definition. Bigger models
| can do more things than smaller models. That to me doesn't make
| the delta between them "emergent".
|
| Not that I've got a better definition...
| vertis wrote:
| The wikipedia page also has an article on large language
| models[0] that includes a section on emergent behaviour.
|
| > While it is generally the case that performance of large
| models on various tasks can be extrapolated based on the
| performance of similar smaller models, sometimes large models
| undergo a "discontinuous phase shift" where the model suddenly
| acquires substantial abilities not seen in smaller models.
| These are known as "emergent abilities", and have been the
| subject of substantial study. Researchers note that such
| abilities "cannot be predicted simply by extrapolating the
| performance of smaller models".
|
| [0]: https://en.wikipedia.org/wiki/Large_language_model
| sampo wrote:
| The reference/source to that Wikipedia paragraph is:
| https://openreview.net/forum?id=yzkSU5zdwD
| vertis wrote:
| The owner of the link is listed on that page, so I'm
| guessing it's no coincidence.
| bravura wrote:
| To me, it seems like a good definition.
|
| If scaling the size of the model leads to a linear increase in
| capability, then nothing is emergent.
|
| If scaling the size of the model accretes almost no gains, and
| then there is a huge unpredicted step in capabilities, a
| capability has "emerged".
| layer8 wrote:
| Emergence doesn't need to depend on scale. For example,
| gliders in the Game of Life are emergent behavior, and they
| don't depend on scale. The way proteins fold is emergent from
| the physical forces, but is not a function of scale, just of
| particular configurations. Emergence is about the level on
| which an explanation or description works. You only "see" the
| glider move at a higher level of description than the
| cellular rules.
| chasd00 wrote:
| I thought the thing that is scaring people is that there
| isn't a defined relationship between model scale and
| capabilities. You may triple the model size and get no
| increase but then quadruple it and get 10,000% increase in
| capability. So no one knows where these massive models are
| headed and people want to pause until they understand the
| relationship.
| Havoc wrote:
| hmm...yes that's perhaps better. Defining it as a bigger
| improvement than would be expected from the size increase.
|
| Don't think the baseline expectation would be linear per se
| though
| kerkeslager wrote:
| The problem isn't that they should have defined "emergent"
| better, it's that they should have used a different word that
| actually means what they intend to communicate, such as "size-
| based" or "scale-dependent".
| cowl wrote:
| an Emergent behavior is usually defined as a behavior that is
| not expected from the training or the starting configuration.
| Their Emergent Definition is absurd. is like saying that
| understanding Calculus is emergent behavior in 12 grade because
| in was not present in 3rd grade.
___________________________________________________________________
(page generated 2023-04-12 23:02 UTC)