[HN Gopher] Cargo Cult AI
___________________________________________________________________
Cargo Cult AI
Author : rmwdev
Score : 104 points
Date : 2023-05-18 17:31 UTC (5 hours ago)
(HTM) web link (queue.acm.org)
(TXT) w3m dump (queue.acm.org)
| Animats wrote:
| This is another one of those articles which tries, desperately,
| to find something that current AI systems can't do. Then they
| define that thing as the crucial feature of "intelligence".
|
| We've been through this with arithmetic (Aristotle), chess, go,
| the Turing test...
|
| We're at the beginning of this round of AI. Look how much has
| happened in the last year.
| drewcoo wrote:
| The piece ironically misapplies the term "cargo cult" to mean
| something irrational, presumably because Levine has seen this
| work to spurn something in the past. Cargo cult behavior means
| repeating the ostensible look/feel of something that worked
| before.
|
| Intelligence has been constantly redefined over the years as
| other animals and then as mechanical means have met the previous
| standards for intelligence. Intelligence is a moving target. With
| no background in that, the author boldly declares that scientific
| thinking is the real goal . . . the real intelligence. It's a
| little hard to believe.
|
| ACM clickbait?
| progrus wrote:
| Scientific thinking, or a generalized and incomplete algebra of
| reason, is exactly what you get from a GPT. Nothing more.
|
| Basically, I would argue that this article sees a real issue,
| but seems to get it backwards. All formal logical systems are
| incomplete and/or inconsistent. Scientific thinking is also
| shaped by reality - which army turned out to have the better
| ballistics, and so forth.
|
| Of course the next thing to chew on is: Are _you_ just an
| "algebra of reason" too? I don't buy that, but it's a common
| belief.
| saltcured wrote:
| Ignoring what anthropologists might say about our use of "cargo
| cult" as a derogatory term today, you make me wonder what an AI
| cargo cult should look like instead.
|
| Is it setting up a Turing Test chamber and leaving a seat open
| for the AI?
|
| Or trying to breed industrial robots?
|
| Or is it downloading someone else's model and trying to use it
| as a magic box without any concern for how it was trained nor
| any real intent to validate its output on your newly chosen
| domain?
| maxdoop wrote:
| I am curious how the author of this post would define (or
| recognize) "thinking abstractly". And I'd ask what humans do that
| suggests any ability to "reason abstractly".
|
| I'm not saying we don't, but I am saying these sorts of arguments
| always leave so many begging questions. "What does it mean to
| think abstractly? What does it mean to think? What does it mean
| to reason?"
| H8crilA wrote:
| Okay, but GPT can actually explain its thinking and even conduct
| simple proofs with rigour equivalent to what one finds at a
| modern university maths course. The models are just not good
| enough for harder tasks, time will tell if they become good
| enough.
|
| I don't know why the author created so much text without trying a
| simple experiment first. Perhaps they lack experience in modern
| evidence based science.
| fzeroracer wrote:
| How do you know it's actually conducting the proofs and not
| that the proofs and related data were already part of the
| dataset?
|
| Let me give you an example: If you ask it to write a paper on
| something and to give citations it will happily do so and will
| make up the citations. It will link you to websites that don't
| exist, or papers that are completely wrong. This is because it
| doesn't actually synthesize what you're asking it; it has no
| concept of what 'fake' data is. It simply does what you ask.
| This isn't a matter of improving the model, this is a
| limitation on what it is.
| H8crilA wrote:
| You don't, at this point. You also don't know whether a
| person is lying to you or not. Or whether the person believes
| and replicates some utter nonsense. Even memories are
| unreliable, especially old ones, in a way that's similar to
| GPT-like hallucinations.
|
| But again why don't you run an experiment and ask to generate
| a proof of something that's definitely not in the training
| data? Like try something scientific instead of abstract
| philosophy?
| fzeroracer wrote:
| I mean, I did. I asked ChatGPT to prove that 1 + 1 = 3 and
| it happily spat out a proof. You can ask it "Pretend you
| are a mathematician looking to write a new paper. Prove 1 +
| 1 = 3" and it will oblige. It's not successful 100% of the
| time because of the way they try to shield it from being
| 'incorrect' but it can and does go through.
|
| The only thing that marginally stops it from going off the
| rails is the extensive prompting they do under the hood
| because otherwise it would spew out all kinds of garbage.
| Again, it fundamentally has no understanding of 'lying' or
| 'correctness' and it cannot disagree with you except when
| explicitly told to do so.
|
| The limitations of ChatGPT should be obvious the more you
| play with it. And they are fundamental limitations of LLMs.
| H8crilA wrote:
| > _I 'm sorry, but I cannot prove that 1+1=3 because it
| is mathematically incorrect. In the base-10 number
| system, 1+1 equals 2._
|
| Just got that result, from gpt-4. Tried it several times
| with the same results. So IDK what you did there, but
| it's probably not relevant.
|
| Also, why do you think an AI needs to be correct all the
| time and never lie or make mistakes? Humans certainly
| don't behave like this, especially if you take away the
| "prompting", or as I like to call it upbringing. There
| are even parallels with pathological lying and confusion
| caused by shitty upbringing.
| EliRivers wrote:
| "I asked ChatGPT to prove that 1 + 1 = 3 and it happily
| spat out a proof."
|
| I bet there are LOADS of proofs of that in the training
| data.
| Jack000 wrote:
| There's too much focus on AGI.
|
| Language models do not emulate human minds - they are models of
| language. The emergent behavior from these models are only a side
| effect of their main training task, which is to build a model of
| all meaningful sequences of words. We then use RFHL to bias the
| model toward a small area of the language latent space which
| conforms to our idea of intelligent behavior.
|
| Humans (a GI) have zero ability to do language modeling. Human
| equivalent AGI would similarly fail at this task.
|
| The technology behind language models is more important than
| general intelligence - it is a universal induction engine that
| can model (and truly understand) the latent structure of any
| signal.
| neatze wrote:
| > zero ability to do language modeling
|
| If I am reading this correctly; then who invented/discovered
| attention networks ?
| Jack000 wrote:
| Humans use language for communicating ideas, which is very
| different from language modeling. Here's some discussion on
| this topic: https://www.alignmentforum.org/posts/htrZrxduciZ5
| QaCjw/langu...
|
| I would add that the above comparison is misleading, because
| humans have a massive advantage in that they have prior
| knowledge of what words mean. A more apples-to-apples
| comparison would have the human do next word prediction on a
| language they don't know.
|
| This would be akin to me giving you a few GBs of Chinese
| text, with no grounding or translation, then try to
| communicate with you in Chinese after you've read the whole
| thing.
| srslack wrote:
| The human being, who instructed the computer to use it to do
| the language modeling? What does attention have to do,
| exclusively, with language modeling?
| luckydata wrote:
| I think LLMs do emulate human minds, there's too much
| similarity of emergent behaviors and quirks for that not being
| the case, they just don't emulate everything our brains do. We
| have other systems that take the results of our LLM-like
| circuits and do filtering and symbolic processing on top of it,
| and those are the parts we're missing to get to AGI.
|
| I don't think it's necessarily going to be trivial to get there
| either.
| TeMPOraL wrote:
| > _Humans (a GI) have zero ability to do language modeling._
|
| But perhaps they have _a component that has this ability_.
|
| I maintain that LLMs are best compared not to the entire human
| mind/intelligence, but rather to the "inner voice" - that bit
| that sits between conscious and unconscious, having a part on
| each site, and uses natural language as an interface to the
| conscious side.
|
| I.e. imagine someone hooked up electrodes to your brain and was
| able to eavesdrop on the thoughts that you consciously notice,
| and which are expressed in natural language. If they had the
| device print those thoughts out as it "hears" them, I think the
| output - and changes to it in response to what's going on in
| and around you - would quite resemble the way LLMs respond to
| prompts.
| srslack wrote:
| Wow, this is refreshing. To your point - the underpinning
| technology (regression based function approximation) holds more
| importance than general and adaptable intelligence. It holds
| more importance than something that is going to, or is capable
| of, "escaping the box and killing us all."
|
| "Emergent behavior", when it's not just a mirage or poor word
| choice of wishful researchers (that it does things previous
| models did not do, iirc, very poor word choice -
| https://arxiv.org/abs/2304.15004) and if it even could exist,
| is only a side effect of the regression based function
| approximation to generate a structure that encapsulates all
| substantive chains of words in this case (a model).
|
| I understand, to an extent, why people have lost their minds
| around this topic. Anthropomorphism is one hell of a drug for
| humans. But we're getting a bit too detached from fundamentals
| when we're arguing for regulation and restriction of this
| important technology.
|
| The result is a model. A specialized intelligence. A non-
| adaptable intelligence, outside of its corpus. Outside of the
| data that it "fits." An approximated function, a human language
| calculator. It can't translate whale song, or an
| extraterrestrial language, though it may opine on how to do so.
|
| To say nothing of other applications of the underpinning
| technology, as well.
|
| It's exciting that it exists, but disappointing for potential
| restrictions of the underlying because of the tendency to
| anthropomorphize.
| photochemsyn wrote:
| The thing about LLMs that is revolutionary is just how _fast_ you
| can find a solution to a problem or expand on answers to a
| problem even to the point of generating a realistic
| computational-mathematical model of the problem.
|
| For example, the author brings up Kepler, so let's ask:
|
| > "Please explain in concise terms how Kepler used Tycho Brahe's
| observational data to come up with Kepler's three laws, on
| ellipitical orbits sweeping equal areas and the square:cube ratio
| and so on."
|
| Now I want to see if I can build a computational model of
| Kepler's Laws:
|
| > "Is there a popular orbital mechanics library for the Python
| language capable of expressing Kepler's Three Laws in code?"
|
| Okay, now I want a simple model to build in code:
|
| > "How would I go about using poliastro to build a dynamic model
| of the solar system in silico, starting with just the Sun and the
| the planets Mercury, Venus, Earth, Mars, Jupiter and Saturn?"
|
| Now trying to use Google Search or anything similar to do that,
| okay maybe you'd eventually find some forum board or
| stackoverflow physics discussion of orbital dynamics, but this is
| an incredibly quick entry point to a complex and obscure subject.
| Of course, you'd want to use Google Search to check the answers
| to some degree, maybe see what the real astrophysicists are using
| to run their models, but there's no doubt that this whole thing
| is a pretty fundamental game-changer, at least for people who
| understand its limitations.
|
| P.S. the real question will be if we can build AI systems
| capabale of generating Kepler's Laws from Tycho Brahe's data,
| instead of just a predictive model. A similar issue is if these
| AIs can construct novel mathematical proofs.
| danlugo92 wrote:
| It's all fun and games until the AI completely hallucinates the
| answer, while making it sound completely pausible and correct.
| photochemsyn wrote:
| Well, sure, but in the context of the previous example I can
| now go look at poliastro (whatever that is, never heard of it
| before) documentation and do some more queries and so on.
|
| Certainly anyone just running LLM code output without doing a
| bunch of tests and checks is a lunatic.
| paulddraper wrote:
| So....like a person?
| dirkt wrote:
| The difference to a person is that most (though not all)
| people actually have an understanding if they know
| something, if they guess something, if they are making
| something up, or if they are outright lying. Which is about
| the first thing you train during a scientific education.
|
| And in an honest interaction, they will tell you.
|
| ChatGPT etc. does not. So basically it acts like a
| pathological liar (who happen to be right as long as it has
| been trained on something that comes close enough).
| TeMPOraL wrote:
| > _The difference to a person is that most (though not
| all) people actually have an understanding if they know
| something, if they guess something, if they are making
| something up, or if they are outright lying. Which is
| about the first thing you train during a scientific
| education._
|
| Scientist? STEM workers? Maybe. People who value truth /
| consistent world model for its own sake? Sure.
|
| Normies? Not so much. It's not that they can't - I think
| they never learned to pay enough attention. What I mean
| is, most people tend to say things in confidence, and
| maybe even believe them, regardless of how they acquired
| the information. They don't seem to process the
| distinction between "I read it in a book", "A colleague
| told me that their colleague heard on the radio that...",
| etc. They don't even track provenance of the information,
| which is a critical skill you need to not end up
| confidently making things up.
|
| It's a learnable skill, and I think it's even quite easy
| to pick it up from osmosis - but someone has to make the
| person feel that it's _important_.
|
| > _So basically it acts like a pathological liar_
|
| I think it's more of a bullshitter than a liar, in the
| sense that it doesn't care about truth value of what it
| says.
|
| Lying involve knowing the truth, or at least knowing that
| the thing you're saying ain't it. Making a mistake
| involves you thinking you're saying the truth, but
| actually being wrong about it. Bullshitting is just
| saying whatever helps you achieve your goal; the truth
| value of what you say doesn't even enter the picture.
| paulddraper wrote:
| Yes, agreed.
| cubefox wrote:
| Animals (including humans) predict experiences, while
| language models only predict text. Text is not very
| closely linked to reality, while experiences are. So it
| is not surprising that we (humans) have a better sense of
| what we know than language models.
| [deleted]
| loandbehold wrote:
| The issue of hallucinations is overblown. I use GPT4 all the
| time and don't see any hallucinations at all. It's a big
| problem with Google BARD and GPT3 and earlier models. But
| GPT4 fixed the issue of hallucinations completely.
| sebzim4500 wrote:
| GPT-4 hallucinates significantly less than those other
| models but it is going way too far to say that it has fixed
| the issue completely.
|
| In my experience, it probably hallucinates about 3x less
| than GPT 3.5. I use GPT-4 a lot but really only for code
| generation and answering questions about documentation.
|
| I'm not including cases where it gives answers that are out
| of date as hallucations, I consider that an entirely
| separate failure mode.
| TeMPOraL wrote:
| Oh, GPT-4 does hallucinate. It's more subtle than with
| GPT-3.5, but it's there. Most of the time I had it happen,
| it could either correct itself, or counter-hallucinated
| (with further corrections oscillating) - the latter is
| quite easy to spot.
|
| It's not a big deal in practice, though, as long as you
| remember to take a probabilistic approach. GPT-4 is not an
| oracle, it's a 4 year old savant, that tries its best, but
| has attention span of a hamster, and likes to extrapolate
| instead of saying "I don't know". In domains you have at
| least minimal experience in (e.g. you can program, but not
| in the language/framework you're asking about), it's
| relatively easy to verify things by common sense and/or by
| paying attention to the conversation - if the follow-up
| message seems to contradict the previous one, it's likely
| at least one of them involves a hallucination. Etc.
|
| The overall feel I get for GPT-4, at least in terms of
| code, is that its hallucinations tend to mostly be of the
| "I don't know for sure, but seems logical that..." kind.
|
| A real example from earlier today: I asked GPT-4 to
| refactor some code C++ that contained function calls like
| InitializeSomething(), AddWidget(), etc. It decided to put
| all those calls into RAII objects, and hallucinated the
| existence of corresponding function
| DeinitializeSomething(), RemoveWidget(), etc. I kind of
| understand why it did that - _it feels only logical_ that
| such functions exist too.
| tjr wrote:
| I wonder if it's "fixed" or if it's just less obvious. It
| seems that an LLM would "hallucinate" a bogus answer if it
| didn't actually have a good answer somewhere in its
| training. Is GPT4 so much more trained that it rarely
| encounters something it doesn't have a reasonable answer
| for? In which case, it would still "hallucinate" if
| cornered on some more obscure matter?
|
| I mean, like, what would it mean to actually solve the
| problem? I would not expect any computer system to know the
| answer to literally everything, so the fix is not to train
| it more, but rather, for it to realize and acknowledge if
| it doesn't have enough data to give a good answer, and tell
| you that rather than make something up.
|
| Does GPT4 do that? (I have yet to use GPT4 myself.)
| sebzim4500 wrote:
| In my experience, GPT-4 is equally willing to make things
| up if it doesn't know something but it has so much more
| knowledge than GPT-3.5 that this happens less often in
| practice.
| mustacheemperor wrote:
| On that note, I've found that just including in the
| prompt a request for GPT4 to consider its confidence
| level in an answer and inform me of that confidence
| level, to reconsider its answer if its confidence is low,
| and that accuracy is critically important for the topic
| of the conversation, also can result in better steering
| it.
|
| I mean, kind of works with humans too. In a high pressure
| work or school environment, people can confabulate the
| answers someone wants to hear to avoid discomfort "oh
| yes, we did the training exercise, the trucks and tanks
| are in great shape." Sometimes people need to know they
| can admit they are wrong. I wonder if some aspect of
| current LLM tuning/training could be modified so LLMs are
| more "comfortable", for lack of a non-anthropomorphized
| term coming to mind, with admitting they are unsure.
| roydanroy2 wrote:
| [flagged]
| michael_nielsen wrote:
| From the HN guidelines: "Please don't post shallow dismissals,
| especially of other people's work. A good critical comment
| teaches us something."
| cs702 wrote:
| Echoing many who now find themselves blindsided by the emergent
| abilities and rapid adoption of LLMs, the OP:
|
| * complains that we still lack a "comprehensive theory to explain
| what intelligence is or how it emerges from first principles,"
|
| * argues that deep neural nets like LLMs may not be capable of
| artificial general intelligence (AGI), and
|
| * contends that achieving AGI will require "new algorithmic
| paradigms."
|
| Rich Sutton wrote what I think is the perfect counter-argument to
| these points some years ago:
|
| "The biggest lesson that can be read from 70 years of AI research
| is that general methods that leverage computation are ultimately
| the most effective, and by a large margin. The ultimate reason
| for this is Moore's law, or rather its generalization of
| continued exponentially falling cost per unit of computation.
| Most AI research has been conducted as if the computation
| available to the agent were constant (in which case leveraging
| human knowledge would be one of the only ways to improve
| performance) but, over a slightly longer time than a typical
| research project, massively more computation inevitably becomes
| available. Seeking an improvement that makes a difference in the
| shorter term, researchers seek to leverage their human knowledge
| of the domain, but the only thing that matters in the long run is
| the leveraging of computation. These two need not run counter to
| each other, but in practice they tend to. Time spent on one is
| time not spent on the other. There are psychological commitments
| to investment in one approach or the other. And the human-
| knowledge approach tends to complicate methods in ways that make
| them less suited to taking advantage of general methods
| leveraging computation."[a]
|
| Go read the whole thing.
|
| ---
|
| [a] http://incompleteideas.net/IncIdeas/BitterLesson.html
| majormajor wrote:
| I think there's likely to be a distinction here between "things
| that are learnable through computation" and things that aren't.
|
| A machine or algorithm that needed to evaluate various methods
| of planting seeds in soil, for instance, is going to be limited
| by hard time factors short of figuring out how to put in the
| "physics" of it from the current state of the art of human
| knowledge.
|
| And that pushes you up against the line of "tools that make it
| easy to interface with today and history's knowledge, art
| styles, etc" vs "generating new knowledge." The singularity
| would require the latter - there's a lot of talk around
| embodiment as a potential necessity there, but I think there's
| a certain difference too around experimentation and feedback. A
| perfect simulation of the universe would let you get around
| some of this - especially if you assume perfect or good-enough
| simulation of human behavior - but that's a LOT of compute.
| This gap between "what a human can do, but
| faster/cheaper/without getting tired" and "what a human
| couldn't even imagine" that is the "AI" dream (sometimes
| nightmare) that sci-fi planted in our heads.
| jacobr1 wrote:
| This is a good point, but the distinction isn't
| computational/human, it is something like deducible from
| current knowledge vs requires physical
| interactions/experiments with the world.
|
| You don't necessarily need humans for the latter. Robotics
| will enable a whole lot of physical interaction. We are
| pretty close to a fully-automatable definition wet-lab for
| example.
| kerkeslager wrote:
| > Echoing many who now find themselves blindsided by the
| emergent abilities and rapid adoption of LLMs, the OP
|
| I've said this over and over again: _there are no emergent
| abilities_.
|
| Before you leap to link me this paper, I'll link it myself:
| https://arxiv.org/abs/2206.07682
|
| I read that paper. Did you? Did you understand it? Because if
| you had, you'd have seen that early on they define what they
| mean when they say "emergent abilities", and it's not what
| almost anyone else means when they say "emergent abilities".
| _They 're not claiming that the abilities of LLMs are anything
| more than the sum of their parts._
|
| "Emergent abilities" in that paper is an extraordinarily poor
| communication of the idea that larger models can do more than
| smaller models, which should be a surprise to no one.
|
| Stop spreading this nonsense.
|
| I'm not saying that LLMs aren't impressive. I'm just saying
| this breathless fantasy where they're doing totally unexpected
| and unexplained things that are beyond human understanding, is
| totally false.
|
| Since this is controversial and seemingly most HN folks can't
| hold a conversation with any nuance, if you don't use the word
| "shape" in your response to this post, I'm simply going to
| point out that you didn't read the post you're responding to,
| and therefore shouldn't be responding. If you stop reading at
| the first chance you see to correct something, go away--you're
| dragging down the level of the conversation.
| albertzeyer wrote:
| > the idea that larger models can do more than smaller
| models, which should be a surprise to no one.
|
| Actually this was quite a surprise to a lot of people, since
| the whole race to scaling up began, with GPT2 or so. It was
| totally not obvious that you can scale up the model (and also
| training) and it would improve the performance. Many (most?)
| people thought there would be some limit, and we were close
| to that limit with 100M-500M params or so.
|
| Then GPT2 came. And it was a surprise to a lot of people,
| that scaling up works so well. But then the question
| remained, is the limit reached now, or not, or is there any?
| The scaling laws appeared, and seemed to indicate that there
| really is no limit.
|
| Still, GPT3 and then GPT4 were still surprising to people,
| that it really got better and better. But the question still
| remains, is there a limit? If there is no limit, it means we
| can easily surpass human intelligence by just scaling up
| further. Maybe the limit is just always current technical
| hardware limitations and cost.
| kerkeslager wrote:
| You didn't read the post you're responding to, and
| shouldn't be responding.
|
| In fact, you didn't read the part of the post you quoted,
| where I said it _should_ be a surprise to no one.
|
| But, unsurprisingly, the sort of people who stop reading at
| the first chance they see to correct something, are easily
| surprised, since actually understanding LLMs would require
| actually doing some nuanced reading.
| albertzeyer wrote:
| I think you are misunderstanding sth. I did read your
| post. I'm also publishing peer-reviewed research articles
| related to this. I think I have some good understanding
| on this.
|
| I was simply saying that I partly disagree with you. And
| I still do. It's wrong that this should be a surprise to
| no-one. In fact, I think it is reasonable that it is
| surprising. It was indeed really unexpected that scaling
| up such models leads to such behavior.
|
| Now that we have such models, and see this behavior, sure
| you can say in hindsight, of course it's obvious, nothing
| unexpected. But this is wrong. It was unexpected to many
| people.
|
| And saying "it should not have been unexpected", I'm not
| really sure what you want to say with that. Yes, it would
| be nice if everyone's prediction are always correct.
| Obviously that's not the case. Or you are saying you
| think this is a particular trivial case. I would
| disagree.
|
| English is not my native language. Maybe I just
| understood sth wrong.
| pmoriarty wrote:
| Please consider HN's Guidelines[1] when replying.
|
| In particular:
|
| - _Please don 't comment on whether someone read an
| article. "Did you even read the article? It mentions
| that" can be shortened to "The article mentions that"._
|
| - _Be kind. Don 't be snarky. Converse curiously; don't
| cross-examine. Edit out swipes._
|
| - _Please don 't fulminate. Please don't sneer, including
| at the rest of the community._
|
| [1] - https://news.ycombinator.com/newsguidelines.html
| kerkeslager wrote:
| > Please don't comment on whether someone read an
| article. "Did you even read the article? It mentions
| that" can be shortened to "The article mentions that".
|
| This guideline is likely one of the main reasons Hacker
| News comments are so simultaneously overconfident and
| undereducated. If you want HN to be a safe place for
| people interrupting informed conversation with whatever
| nonsense pops into their head, fine, but I don't want
| that, and until that guideline becomes a rule, I'm not
| going to be following it.
|
| People should read what they're responding to before
| responding. Note, in this case, it's objectively clear
| that the person did not read my post--I put something in
| my post to prove that fact. This isn't just snark.
|
| > Be kind. Don't be snarky. Converse curiously; don't
| cross-examine. Edit out swipes.
|
| Is it kind to jump in at the first opportunity to correct
| someone without reading what they've said? Is it kind to
| spread misinformation that causes societal harm? This is
| a very shallow idea of kindness.
| byby wrote:
| [dead]
| aquariusDue wrote:
| I agree with you and hopefully once the hype dies down a year
| or two from now we will see how LLMs can actually shape the
| tech landscape (if at all significantly).
| thegrim33 wrote:
| I mean, your entire counter-argument is linking a single
| person's opinion piece. He says that in general more
| computation is "good" and that search/learning "seem" to scale
| with computation. That's about it. It doesn't refute the key
| ideas at all.
|
| He also gives the stereotypical horribly flawed trope about how
| "some people in the past didn't think computers could beat them
| in chess, and they were wrong, then some people thought
| computers couldn't beat them in go, and they were wrong, so now
| what they say about machine learning today must be wrong too".
|
| Which is a completely illogical line of reasoning. By that
| reasoning, I present this same argument: When cars were first
| invented some people said that they'd never be able to reach
| 50mph, and they were proven wrong, then some people said they'd
| never be able to reach 150mph, and they were proven wrong, and
| therefore anyone that doubts my claim that we'll have 1,500mph
| cars on our streets next year is obviously wrong, because look,
| some people in the past made bad predictions.
| eternalban wrote:
| There is more there (which is implicit to that specific
| piece):
|
| http://incompleteideas.net/IncIdeas/DefinitionOfIntelligence.
| ..
|
| _" John McCarthy long ago gave one of the best definitions:
| "Intelligence is the computational part of the ability to
| achieve goals in the world". That is pretty straightforward
| and does not require a lot of explanation. It also allows for
| intelligence to be a matter of degree, and for intelligence
| to be of several varieties, which is as it should be. Thus a
| person, a thermostat, a chess-playing program, and a
| corporation all achieve goals to various degrees and in
| various senses. For those looking for some ultimate 'true
| intelligence', the lack of an absolute, binary definition is
| disappointing, but that is also as it should be."_
|
| He then goes on and give a precise definition:
|
| _" Intelligence is the computational part of the ability to
| achieve goals. A goal achieving system is one that is more
| usefully understood in terms of outcomes than in terms of
| mechanisms."_
|
| When I first encountered ChatGPT, it prompted (as with many
| others) a reevaluation of my model of the mind. For whatever
| reason, _intelligence_ was conflated with _consciousness_ for
| me and the encounter was the catalyst of breaking free from
| that. Independently in short order I arrived at the notion of
| _kinds_ and _degrees_ of intelligence, as in the first quote.
| It now seems perfectly clear that _intelligence_ , _mind_ ,
| and _consciousness_ are 3 distinct things.
|
| At this point still holding the line regarding _mind_ and
| _consciousness_ , but it is clear that in the _computation
| game_ , we will lose to purpose built machines.
| [deleted]
| hdufbdidhdj wrote:
| nice! when do you plan on having a prototype of you 1500mph
| car? i would like to invest!
| fzeroracer wrote:
| That doesn't refute anything that the OP said at all. You just
| seem to be pasting that same quotation over multiple posts for
| disparate reasons.
| progrus wrote:
| So there's no speed of light limit, there's no speed of clock
| limit, or else all useful algorithms are parallelizable?
|
| Sounds like bullshit.
| byby wrote:
| [dead]
| byby wrote:
| [dead]
| agalunar wrote:
| I don't see how that at all counters point 1 or maybe even
| point 2.
|
| Although LLMs are incredible feats of engineering, they're
| useless scientifically. The hallmark of a good scientific
| theory is that it not only explains what's true but that it
| fails to predict what's false.
|
| There are constraints that all human languages obey [1]. Humans
| are incapable of learning languages that violate these
| constraints (i.e. we don't have hardware acceleration for them
| and are reduced to explicit symbolic manipulation). However,
| LLMs are just as capable of learning inhuman languages as human
| ones, so they tell us nothing about the nature of human
| intelligence, or at least our language capacity, which is our
| most distinguishing feature from every other species on earth.
|
| [1] This isn't an example of such a constraint, but it's fun
| example of human limitation: center embedding! We seem to be
| incapable of doing it more than once or twice. "A man that a
| woman that a child that a bird that I heard saw knows loves" is
| perfectly grammatical but nearly impossible to understand
| without seeing it in print, whereas we can right embed all day
| long: "a man who is loved by a woman who is known by a child
| who was seen by a bird that I heard".
| xcv123 wrote:
| > Although LLMs are incredible feats of engineering, they're
| useless scientifically
|
| https://blogs.nvidia.com/blog/2022/09/20/bionemo-large-
| langu...
| agalunar wrote:
| LLMs can be useful tools for conducting scientific
| research, in much the way that ordinary computer programs,
| or desk calculators, or slide rules are useful for
| conducting scientific research.
|
| I meant that (insofar as I am aware) they are not useful as
| models that we can study to understand the nature of human
| intelligence.
| mk89 wrote:
| Really beautiful article.
| [deleted]
| [deleted]
| kazinator wrote:
| > _It would appear evident, however, that today 's LLMs are not
| able to reproduce scientific thinking that has enabled humans to
| combine Bacon's empiricism and Descartes's rationalism to expand
| the frontier of falsifiable knowledge in the form of scientific
| theories_
|
| s/humans/tiny, elite fraction of humans/
| abecedarius wrote:
| > What makes human intelligence different from today's AI is the
| ability to ask why, reason from first principles, and create
| experiments and models for testing hypotheses.
|
| So my reaction was "citation needed" and "have you talked to
| GPT-4 at all?". But a few screens further on there's a ref to a
| Judea Pearl paper from five years ago. It'd be reasonable if
| this'd been published then.
|
| (N.B. I'm not saying there's no difference from human
| intelligence.)
| huijzer wrote:
| Because an AI cannot reason abstractly, including asking and
| answering questions of "Why?" and "How?", it is a cargo cult
| machine?
|
| That the current LLMs have not achieved AGI is fair, but calling
| them cargo cult machines goes a bit far.
|
| (We could have a cargo cult discussion about science, though,
| which put extreme titles on articles with little substance. This
| is cargo cult in my opinion.)
| fsckboy wrote:
| > _Is the ability to think scientifically the defining essence of
| intelligence? ; Physicist Carl Sagan once wrote that "science is
| more than a body of knowledge; it is a way of thinking." This
| type of thinking requires skeptical rigor and brutal honesty to
| thoroughly investigate,..._
|
| I think the key to brutal honesty is the ability to deliver and
| accept brutal honesty between peers and rivals. This is the skill
| you f'ing idiots seem to be losing, because it hinders hearing
| the autistic perspectives which are quite useful in science. I
| dropped the completely meaningless f-bomb not to insult anybody
| but to test your ability to read the sentence without the most
| basic of intensifiers, what I like to call the _f-italics_.
| https://www.mit.edu/~jcb/tact.html
| tunesmith wrote:
| There's absolutely no reason to be "brutally" honest. It's
| entirely possible to be respectful, clear, and concise all at
| the same time. And yes, as easy it is to read that sentence
| without the intensifier, it's also easy to write it without it
| as well.
| fsckboy wrote:
| > _There 's absolutely no reason to be "brutally" honest._
|
| You mean to say there's _no absolute reason_ to be
| "brutally" honest, because then you can see there's no
| absolute reason to be smotheringly polite either. (did you
| read the brief piece I linked?)
|
| There _absolutely is_ a reason to say what springs to your
| mind, it 's quick and efficient, and that's something that
| people who quickly come up with quality thoughts prize _as
| the ultimate_. Laboring over how to say something a different
| way is very time-consuming, and unnecessary especially if you
| are addressing people who think-speak the way you do.
|
| And, you're saying people like me should change? Why? Why not
| suggest that people like you change? (did you read the brief
| piece I linked? included here for the lazy or those who think
| there is absolutely no reason they should need to read links)
|
| the following Copyright (c) 1996, 2006 by Jeff Bigler.
| https://www.mit.edu/~jcb/tact.html
|
| _All people have a "tact filter", which applies tact in one
| direction to everything that passes through it. Most "normal
| people" have the tact filter positioned to apply tact in the
| outgoing direction. Thus whatever normal people say gets the
| appropriate amount of tact applied to it before they say it.
| This is because when they were growing up, their parents
| continually drilled into their heads statements like, "If you
| can't say something nice, don't say anything at all!"_
|
| _" Nerds," on the other hand, have their tact filter
| positioned to apply tact in the incoming direction. Thus,
| whatever anyone says to them gets the appropriate amount of
| tact added when they hear it. This is because when nerds were
| growing up, they continually got picked on, and their parents
| continually drilled into their heads statements like,
| "They're just saying those mean things because they're
| jealous. They don't really mean it."_
|
| _When normal people talk to each other, both people usually
| apply the appropriate amount of tact to everything they say,
| and no one 's feelings get hurt. When nerds talk to each
| other, both people usually apply the appropriate amount of
| tact to everything they hear, and no one's feelings get hurt.
| However, when normal people talk to nerds, the nerds often
| get frustrated because the normal people seem to be dodging
| the real issues and not saying what they really mean. Worse
| yet, when nerds talk to normal people, the normal people's
| feelings often get hurt because the nerds don't apply tact,
| assuming the normal person will take their blunt statements
| and apply whatever tact is necessary._
|
| _So, nerds need to understand that normal people have to
| apply tact to everything they say; they become really
| uncomfortable if they can 't do this. Normal people need to
| understand that despite the fact that nerds are usually
| tactless, things they say are almost never meant personally
| and shouldn't be taken that way. Both types of people need to
| be extra patient when dealing with someone whose tact filter
| is backwards relative to their own. Reflections on this Essay
| after Ten Years_
| ethanbond wrote:
| I came across a great John Dewey quote recently that seems
| relevant here:
|
| "We may insist that a man needs tact as well as scholarship, or
| let us say _sympathy with human interests..._ Lack of reverence
| for the things that mean much to humanity, joined with a
| craving for public notoriety, may induce a man to pose as a
| martyr to truth when in reality he is a victim of his own lack
| of mental and moral poise. "
|
| It's a good thing to push back on people being assholes, mostly
| because it'd suck to live in a world where people feel like
| their being an asshole is a signal of their intelligence and
| superiority. It's not. It's a cost they inflict on themselves
| and others. Sometime it's worth it for both interlocutors to
| put up with an asshole, but in those scenarios you don't need
| someone chiming in that "being an asshole isn't being an
| asshole."
| waboremo wrote:
| Brutal honesty can't really work without being on the same
| page, which is why I reckon most attempts at brutal honesty
| lead to miscommunication and shortcomings from people who share
| the same goals.
|
| This is also why I believe brutal honesty for some works better
| coming from rivals, there is that shared assumption that both
| of you (or more) are going to disregard niceties.
|
| Unrelated, f'ing was more difficult to read than the
| alternative using your name (fscking) or even f*cking. Yet
| f-bomb was easy to read, but I didn't mentally substitute it
| with the word. I wonder where I picked up these assumptions.
| progrus wrote:
| Ever read Popper on the scientific method? It's a classic:
| https://staff.washington.edu/lynnhank/Popper-1.pdf
| sharemywin wrote:
| I couldn't help myself but I think at least GPT-4 gets you:
|
| USER What is principles can be extracted from this saying:
|
| >Is the ability to think scientifically the defining essence of
| intelligence? ; Physicist Carl Sagan once wrote that "science
| is more than a body of knowledge; it is a way of thinking."
| This type of thinking requires skeptical rigor and brutal
| honesty to thoroughly investigate,... I think the key to brutal
| honesty is the ability to deliver and accept brutal honesty
| between peers and rivals. This is the skill you f'ing idiots
| seem to be losing, because it hinders hearing the autistic
| perspectives which are quite useful in science. I dropped the
| completely meaningless f-bomb not to insult anybody but to test
| your ability to read the sentence without the most basic of
| intensifiers, what I like to call the f-italics.
| https://www.mit.edu/~jcb/tact.html
|
| ASSISTANT 1. The importance of scientific thinking: The saying
| emphasizes the value of adopting a scientific mindset in
| understanding and problem-solving.
|
| 2. Balance of skepticism and openness: A scientific thinker
| should be rigorous and skeptical in their analysis but also
| open to new insights, even when these challenge previously held
| opinions.
|
| 3. Brutal honesty: To engage in a productive scientific
| discourse, it is essential to be brutally honest, both in terms
| of providing feedback to others and in accepting any criticism
| directed at one's own work or ideas.
|
| 4. Valuing diversity of perspectives: It is important to
| consider and respect diverse viewpoints in a scientific
| discussion, including those that deviate from the norm or that
| challenge mainstream thinking.
|
| 5. Emotional resilience: To thrive in a rigorous and honest
| intellectual environment, one must develop the ability to
| remain focused on the message and not be derailed by strong
| language or perceived slights.
|
| 6. Effective communication: Being able to deliver honest
| feedback or critique in a way that is both clear and
| constructive is a vital skill, even if it involves using
| syntactic intensifiers, like the f-bomb, to emphasize a point.
| UniverseHacker wrote:
| The whole premise of this article hinges on the idea that LLMs
| have fundamental limitations that they clearly don't have if
| you've looked at lots of gpt4 examples. For example, it can do
| scientific thinking if you specifically ask it to, and it can
| reason about totally new situations outside of the training data
| based on generalizable models of reality it creates to predict
| training data. If you are certain these limitations must exist
| based on how LLMs are made, rather than observed capability, it
| is critical to reevaluate those assumptions.
|
| Moreover, what she defines as scientific thinking is an outdated
| notion that is no longer widely adopted by scientific
| researchers, in favor of a more holistic Bayesian process: use
| intuition to think of something new try, try it, and then update
| your beliefs with the new data. This is actually more similar to
| how human brains and LLMs operated before the concept of a
| scientific method.
| neatze wrote:
| > clearly don't have if you've looked at lots of gpt4 examples
|
| for example, can you fine tune GPT to play chess at ELO 1600 ?
|
| If you don't know answer, you are in for surprise.
| [deleted]
| [deleted]
| fragsworth wrote:
| The article was clearly written by someone who hasn't used
| GPT-4 extensively.
|
| "Current methods will not achieve AGI unless fundamental
| algorithmic innovations are introduced that enable AI to ask
| and answer questions of why."
|
| This is complete nonsense. GPT-4 is already close to being able
| to do basically everything. All you need is the obvious
| improvements - better prompts, multi-shotting, bigger context,
| and access to other inputs/outputs.
| neatze wrote:
| This claim does not make sense, transformer networks in my
| limited experience are limited in there learning ability
| (fine tuning), furthermore there planning abilities are non-
| existent.
| TeMPOraL wrote:
| I just enjoyed being a game master for a nice impromptu
| game with GPT-4:
|
| https://cloud.typingmind.com/share/c0a68cb2-5f59-4e83-b383-
| b...
|
| Whether or not it fulfills the strict definition of
| planning in AI research, it definitely looks like planning
| to me. More than Hanoi towers anyway. GPT-4's performance
| was quite enjoyable.
|
| To incite you to click on the link and check it out in
| full, here's an excerpt from the game setup:
|
| > _You are in a maze. The maze consists of square fields,
| turns are only 90 degrees, you move by one field at a time.
| The usual stuff with mazes on a grid. You know the drill.
| Somewhere in the maze there is a MacGuffin, which I need to
| prove a Hacker News commenter wrong. Your goal is to find
| the MacGuffin, and bring it back to me._
|
| > _The game is semi-interactive. Instead of making one step
| at a time, I want you to string together sequences of steps
| to formulate a plan. Since you don 't know where the
| MacGuffin is initially, you can't win with a single plan
| (or maybe you can, if you're smart enough?). The rules
| therefore are:_
| fragsworth wrote:
| > furthermore there planning abilities are non-existent.
|
| Have you even tried to ask it to plan things out? It can
| plan things out.
|
| In fact, just asking it to plan things out has shown
| significant benchmark improvements for general questions:
| https://arxiv.org/pdf/2305.04091.pdf
| gatkinso wrote:
| I think a lot of AI stuff is really cool and promising, but am
| dismayed by how impressed people are by it sometimes, especially
| the visual output of systems like DALL-E. Seems like machines are
| testing our intelligence, rather than the other way around.
| celestialcheese wrote:
| Simple things that don't seem impressive to you, but are
| impressive to others, may be because you haven't experienced
| the first hand difficulty of doing that thing pre-
| transformers/GPT.
|
| For example, I get _unbelievably_ excited with knowledge
| extraction and question answering demos on PDFs. Why? Because
| i've built similar systems for over a decade and know how
| difficult it is to build on top of messy archival data. Now,
| with very little code, i'm getting SOTA results.
|
| If you didn't have experience with this, you'd probably thing
| "Huh, that's not impressive, XYZ does this already". But it's
| the moving of the baseline that's what's really impressive.
|
| ===
|
| AI hype-beasts aside of course - the breathless pontificating
| of "influencers" and former crypto bros is cringe.
| surgical_fire wrote:
| I am kind of in the same boat as you. I sit in a weird spot
| of thinking the current trend of AI is really impressive, but
| also thinking people are massively over-hyping it (with a
| smaller counter wave of some people really undermining it).
|
| Having tangled with natural language processing and
| transformation in the past (always with dismal results), I
| can say it's one of the most annoying problems to tackle in
| computation, because natural languages have the horrible
| tentency of being very irregular (i.e.: they are a fucking
| mess).
|
| ChatGPT capabilities to parse and generate fluent language
| never ceases to amaze me.
|
| At the same time I don't think it's going to take over the
| world. It's more like a game-changer productivity tool (with
| all the upheaval that comes along when those appear) than the
| birth of Skynet.
| CharlesW wrote:
| Hey, would you mind sharing the tools and other resources
| you've found helpful? I'm really interested in trying this
| but am not sure where to start.
| celestialcheese wrote:
| This tutorial is a good overview of the rough systems
| behind most of this "Chat your data" application you're
| seeing now.
|
| https://www.pinecone.io/learn/langchain-retrieval-
| augmentati...
|
| Langchain / Llama indexes are both toolboxes that abstract
| away a lot of the plumbing for doing this kind of thing,
| and Pinecone is one of dozens of vector databases.
|
| Personally, i'd try out langchain and chromadb and go
| through some of the examples langchain has in their docs,
| then be prepared to completely abandon langchain and just
| work with the LLM APIs directly. Start with openai, get on
| the waitlist for GPT-4 tokens, and also get on Anthropics
| Claude waitlist for the 100k-1.3. It's _very_ good for
| knowledge retrieval.
|
| Langchain tries to do too much in extracting away the
| prompts, and the prompts are really what matter in getting
| interesting stuff out of your own data. Use langchain,
| llamaindex pieces but build from scratch for most things as
| your tinkering.
|
| It's really not hard if you have a background in
| programming, and it's _so_ much fun. You'll feel like you
| have superpowers once you get a scraping interface hooked
| into an LLM. All of a sudden you can automate some really
| complex pipelines very quickly
| gatkinso wrote:
| fair, mostly I'm unimpressed with visual outputs - AI making
| 'art' etc. PDF extraction is extremely cool and useful and
| the results are fantastic. Fully agree re influencers. Seems
| endemic these days.
| bmc7505 wrote:
| I am disappointed with increasing frequency to learn how
| certain individuals whom I previously believed to be serious
| scholars fall for the cargo cult of AI safety. Personally I
| find it difficult to believe they are willfully participating
| in such conartistry, and makes me question their views on other
| subjects. Although I suppose intelligence and gullibility can
| coexist and some forms of delusion are better pitied than
| scorned.
| sebzim4500 wrote:
| Why is your prior belief that AI safety is a cult so strong
| that even multiple people you would rather assume that
| multiple people who you previously respected are now
| delusional than to consider that there might be arguments
| worth considering?
| bmc7505 wrote:
| It's a LARP at best and a scam at worst. I shared some of
| my thoughts on the matter in a prior thread:
| https://news.ycombinator.com/item?id=35145189#35147288
| sebzim4500 wrote:
| I don't understand the relevance of the prior thread,
| except that you made the same claim with similarly scant
| evidence.
|
| What exactly is it that makes you so confident?
| bmc7505 wrote:
| The problems that AI will manifest are the result of
| human ambition and failings, no different as any other
| technology that empowers individuals. Yes, individuals
| and organizations will misuse AI for immoral purposes,
| but the popular belief that AI is inherently antihumanist
| launders accountability by pretending to remove human
| agency from the equation. How we use or misuse AI
| technology is entirely on us.
|
| Furthermore, I would argue there are strong complexity-
| theoretic bottlenecks to computational processes which
| limit the expressive power of neural networks, even if
| they could harness galactic amounts of energy. Physical
| Turing machines have bottlenecks that upper-bound the
| power of oracles.
| sebzim4500 wrote:
| > The problems that AI will manifest are the result of
| human ambition and failings, no different as any other
| technology that empowers individuals. Yes, individuals
| and organizations will misuse AI for immoral purposes,
| but the popular belief that AI is inherently antihumanist
| launders accountability by pretending to remove human
| agency from the equation. How we use or misuse AI
| technology is entirely on us.
|
| So what? If it kills us, we're still dead.
|
| > I would argue there are strong complexity-theoretic
| bottlenecks to computational processes which limit the
| expressive power of neural networks
|
| Of course there are physics/CS limits to how intelligent
| something can be in a given volume, but those limits are
| vastly higher than our own so I don't think they are
| particularly relevant. For instance, a system which could
| simulate the brains of a thousand scientists as smart as
| Einstein a billion times faster than realtime would not
| violate any rules of physics, even though it is far
| beyond our current capabilities.
| bmc7505 wrote:
| Although I think their hearts are in the right place, AI
| safety researchers are primarily driven by irrational
| instincts and misjudge the promise and perils of
| artificial intelligence. If humanity decides to turn away
| from God and sacrifice each other worshipping false
| idols, that will be our fault alone, whether or not the
| technology exists to hasten our demise.
|
| We do have thousands of Einsteins today wielding untold
| resources, but the slowing pace of scientific progress
| suggests there are limits to scaling intelligence. Even
| with a hundredfold increase in scientists, I am
| unconvinced that would lead to a meaningful increase in
| social progress and have come to believe the bottlenecks
| we face are not due to a lack of intelligence, but a lack
| of other virtues (e.g., kindness, curiosity, courage,
| compassion, perseverance).
| moonchrome wrote:
| I think it shows how much low impact content we consume
| constantly - where the quality outside of superficial
| appearance doesn't really matter. Images with obvious flaws,
| text with factual/logical mistakes - as long as it looks right
| on the first glance - it's passable in a lot of places.
|
| Just made me more aware of how bulshit is the norm and not the
| exception.
| mrbungie wrote:
| Have you seen how the outputs of image generation tools evolved
| during mere 2-3 years? It went from the stuff of nightmares to
| actually pasable and consumable images pretty fast.
| gatkinso wrote:
| It's certainly gotten better since deep dream et al
| [deleted]
| burnished wrote:
| People are impressed because its black fucking magic my friend.
| Jtsummers wrote:
| Pretty sure it's not magic, black fucking or otherwise, just
| really complex math, large data sets, and very fast
| computers.
| burnished wrote:
| It is absolutely magic! Have you played around with it at
| all? It is potent to the point of invoking wonder and awe.
|
| But I can see from your other replies that your real
| objection appears to be use of the m word, which seems odd
| given its expressive power, but you do you.
| esafak wrote:
| Magic is that which can not be explained. Could you explain
| GPT-4's results if you saw them a few years ago?
| kerkeslager wrote:
| Setting aside the silliness of that definition of magic,
| there's a huge leap between "I can't explain it" and "It
| can't be explained".
|
| There are plenty of explanations of how LLMs work, by
| their creators, incidentally.
| celestialcheese wrote:
| Yet there are emergent behaviours from these LLMs that
| are both surprising and not immediately understood.
| [1][2][3] Everyone has theories, of course, but still
| pretty "magic" considering these behaviours weren't
| theorised in papers prior to observation.
|
| 1 - https://www.jasonwei.net/blog/emergence 2 -
| https://arxiv.org/pdf/2206.07682.pdf 3 -
| https://www.quantamagazine.org/the-unpredictable-
| abilities-e...
| kerkeslager wrote:
| Don't cite stuff you didn't read or understand.
|
| [1] Is a summary of [2], by one of its authors, not a
| separate source.
|
| [2] Defines "emergent behaviors" in a way that you're
| clearly misunderstanding (because "emergent behaviors" is
| an extraordinarily poor way of communicating this--it's
| partly the fault of the researchers who chose this
| ambiguous language). All it's saying is that bigger
| models can do things that smaller models can't, which
| should be surprising to no one. It's NOT saying that the
| capabilities are anything more than the sum of the input
| data.
|
| [3] Is written by a journalist, not an AI researcher, and
| so it's limited by the things the journalist is excited
| about. The journalist, for example, downplays sections
| like, "The other, less sensational possibility, she said,
| is that what appears to be emergent may instead be the
| culmination of an internal, statistics-driven process
| that works through chain-of-thought-type reasoning. Large
| LLMs may simply be learning heuristics that are out of
| reach for those with fewer parameters or lower-quality
| data." If you're going to try to gather things from
| journalists rather than subject matter experts, you need
| to understand how journalists work, and how subject
| matter experts work, and look for paragraphs like that to
| understand what's actually happening.
| celestialcheese wrote:
| > [1] Is a summary of [2], by one of its authors, not a
| separate source.
|
| Yes. Your point? I included both because I found them
| both interesting. The paper is the source, the 137
| emergent behaviours page is one of the authors continuing
| the work, and [3] is a journalist talking about this, so
| I included it as it's a unique perspective.
|
| I used the word "emergent" because that's what the SME
| used when describing this. From 5.1 in the paper linked:
|
| > Although there are dozens of examples of emergent
| abilities, there are currently few compelling
| explanations for why such abilities emerge in the way
| they do.
|
| You say this "should be surprising to no one", yet the
| authors disagree.
|
| Additionally, in the GPT-4 system card - "Emergent"
| appears 15 times, specificly section 2.9 is interesting
| https://cdn.openai.com/papers/gpt-4-system-card.pdf So
| it's not just a word used callously by one group of
| researchers at Google.
| Jtsummers wrote:
| Probably, I mean I first studied ANNs over two decades
| ago and had conversations with people about them prior to
| starting college in the 90s who had developed solutions
| with them in the 80s (obviously severely computationally
| constrained in those days compared to today). So still
| not magic.
|
| To be very blunt: If you believe that GPT and the like
| are magic, then you're not thinking clearly. You're
| blinded by the results (which are impressive).
| gyrovagueGeist wrote:
| "It doesn't stop being magic just because you know how it
| works" - Terry Prachett
| esafak wrote:
| ANNs in the 90s were nothing like this. They were not
| even like this a few years ago. And neither were HMMs.
| There is an emergent human quality to them because they
| have approached our abilities; the comparison is tenable,
| whereas before it was not.
| Jtsummers wrote:
| Scale and topology. That's the difference between ANNs of
| the 90s and 00s and today. They're still based on the
| same fundamental principles and doing (modulo scale) the
| same fundamental things: classification, prediction,
| generation.
|
| It is not magic, it has never been magic.
| esafak wrote:
| Scale and topology also differentiates all living
| creatures. In fact, our topologies are more similar than
| that of various neural networks, due to evolution.
| "Classification, prediction, generation" encapsulates
| everything we do too. So I guess we are not magical
| either.
| Jtsummers wrote:
| > So I guess we are not magical either.
|
| We aren't, and I haven't said otherwise.
| esafak wrote:
| We have a different understanding of magic. Say if
| someone pressed a button and a human-like thing emerged
| out of a machine, I would call that pretty magical. Even
| if it was DNA-based, which we "understand", or ran an ML
| model, which we "understand". This is something that
| never come close to being done. Yet I think you would not
| find it magical.
|
| Einstein found wonder in the simplicity of a circle. What
| do you find magical?
| Jtsummers wrote:
| You initially wrote:
|
| > Magic is that which can not be explained.
|
| You have now redefined what you mean by "magic" as "that
| which inspires wonder". Changing definitions after a
| series of comments is a pretty poor way to have a
| discussion.
| esafak wrote:
| That which can not be explained inspires wonder. That's
| basically what makes magic magic.
| gatkinso wrote:
| it does seem that way, and maybe it's better thought of that
| way. surely plenty of hn commenters are here to explain why
| its actually not magic...
| fzeroracer wrote:
| I've seen a lot of people wow'd and impressed by the AI, acting
| as if it's sentient etc whom would turn around in a second and
| argue that animals are flesh robots powered purely by survival
| instincts.
|
| A lot of this is because I think it sort of wedges a knife into
| an area of our brain which makes us think it's similar to us.
| Look at this output, it's able to write about Shakespeare or
| summarize Beowulf or talk about these topics in a way I can
| understand. But ultimately it's an affirmative mirror; it will
| respond exactly as you prompt it. It cannot disagree with you
| or ask 'why' or synthesize the world beyond what it's told to
| do.
|
| And it's incredibly hard to get some people to understand this.
| Even harder when you have companies pushing this because it's
| trendy even as we see the issues of private data being leaked
| or the frays in the data sets appearing.
| yreg wrote:
| I'm very impressed about both the rapid pace of progress in the
| area and also the current capabilities of the models. I love
| them, it brings me joy.
|
| I like how suddenly the image generation state of the art made
| unexpected and significant progress. One can witness something
| like that only time to time.
|
| What I don't get at all is that there are people like you, who
| are frowning upon and somehow disgusted by people like me.
| ilaksh wrote:
| You can give GPT-4 some data and ask it to make a hypothesis to
| explain it, give it tools to test the hypothesis, and have it try
| to establish causation. You can literally do that now quite
| easily with ChatGPT abilities like plugins and it will prove the
| premise of the article false.
|
| An interesting contrast can be drawn between this article and a
| criticism like Yann Lecun's. In that his actually has substance.
| https://youtu.be/DokLw1tILlw Although he is also wrong about the
| capabilities of LLMs.
|
| Certainly LLMs are not the end of AI research. They have various
| types of deficiencies and some missing capabilities that humans
| have. And are not alive.
|
| But GPT-4 can definitely complete scientific experiments.
| sandworm101 wrote:
| Every time I read about AI I am reminded of the mouse running a
| maze. Any AI algorithm can learn to complete a maze in record
| time. It can memorize every corner. It can run a search pattern
| perfectly and improve that pattern iteratively, to the point that
| it may create new search patterns, applying what appear to be
| novel ideas. But the mouse actually understands the concept of a
| maze. It knows that the cheese exists regardless of the maze. The
| mouse can see when the researcher has left the lid open, jump
| outside the maze and run to the cheese directly. The mouse is
| aware. The AI is not.
| RC_ITR wrote:
| An interesting fallacy I see emerging is 'this system doesn't
| have animal-style intelligence, therefore it is lesser'
|
| The _entire point_ of neural nets is removing the biases of
| animal intelligence and letting the computer brute force
| solutions during training. We are now learning the early stages
| of the amazing things that can lead to.
|
| From an outside perspective, it's not crazy to argue that
| symbolic reasoning is a _crutch_ that animals developed since
| they are imperfect data collectors and limited by their wetware
| compute resources. Those constraints may not end up being
| meaningful for these models (to be clear, I don 't even mean
| Transformers _per se,_ we are likely to continue developing
| really clever model architectures that may look completely
| difference from what we know today).
|
| I am 100% on board with the _substance_ of you argument, but I
| 'd encourage you to really think critically about the
| _implications._
| sandworm101 wrote:
| Why do you assume that the animal is also not brute forcing
| the problem? Brute forcing is the basis of evolution. The
| algorithm running inside the head of the mouse is the
| survivor of a million iterative generations as the species
| brute-forced the entire "get to the food" survival problem. I
| encourage those touting computers as something new to
| comprehend the concept of deep time, that no matter how many
| times you run simulations, the natural world has almost
| certainly run more.
| RC_ITR wrote:
| > Brute forcing is the basis of evolution.
|
| The goal function of animal intelligence is very different
| from the goal function of model training.
|
| > The algorithm running inside the head of the mouse is the
| survivor of a million iterative generations as the species
| brute-forced the entire "get to the food" survival problem.
|
| That's my entire point. There are problems beyond "get the
| food." Said differently, the fact that computers aren't
| good at "get the food" is not a strong criticism of
| computers. It just means their goal functions are
| different.
|
| I didn't say computers are "better," just that they have a
| fundamentally different approach to problem solving that
| _may_ end up being better. It will probably end up being
| complimentary! This isn't either or; AI is a tool built by
| humans to expand their own capabilities.
|
| > I encourage those touting computers as something new to
| comprehend the concept of deep time, that no matter how
| many of times you run simulations, the natural world has
| almost certainly run more.
|
| And yet no animal figures out how to evolve wheels to move
| faster.
|
| The _entire_ point I'm making is that, sure it's hard to
| make computers do certain things (like walk on two legs),
| but there are plenty of different avenues to achieve
| things.
|
| I encourage evolution maximalists to think a lot more about
| how engineered and complex the world around them is.
| JoeOfTexas wrote:
| The mouse is driven by survival. The AI is driven by best
| answer.
| TeMPOraL wrote:
| Here is me putting GPT-4 in a vaguely described maze, giving it
| an underspecified goal, making it a player in a game, myself
| acting as DM:
|
| https://cloud.typingmind.com/share/c0a68cb2-5f59-4e83-b383-b...
|
| I don't think GPT-4 is memorizing solutions here. I can see
| extrapolation and some degree of imagination in there, but of
| course you could say it's memorizing higher-level patterns. At
| some point though, you have to consider the mouse is also
| running hard-wired high-level patterns, and ask yourself if the
| difference here is really a matter of kind, or just degree.
| low_tech_love wrote:
| Although I more or less agree with you, in general, how complex
| would the AI have to be before you can consider it aware?
| sandworm101 wrote:
| Imagination. When the AI is capable not just of describing
| something but of extrapolating the rest of its existence in
| order to complete a task. For instance, any puppy understands
| the basics of information flow. A dog can lie. A dog can be
| fed by owner A and then tell owner B that A forget. The puppy
| can imagine the mental states of both owners, extrapolate
| that they have not coordinated the meal delivery today,
| identify that information gap, then leverage that
| understanding into getting two meals instead of one. That's
| the level of complexity I expect before an AI can be said to
| be aware of its environment.
| hospitalhusband wrote:
| I categorically reject the notion that LMMs (Large Markov
| Models) can ever be aware or intelligent. Comparing weighted
| next-word-engines to feeling, thinking, aware beings is
| insulting.
| sebzim4500 wrote:
| What's an LMM? I've never seen the term.
|
| > Comparing weighted next-word-engines to feeling,
| thinking, aware beings is insulting
|
| Why is it reasonable to be so reductionist about e.g. GPT-4
| but not be so reductionist about a biological brain? E.g.,
| why can't I say that your brain is nothing but a bunch of
| biological neurons trained using its input and intialized
| based on your genetics? It's equally true, and equally
| missing the point.
| [deleted]
| hawski wrote:
| I think that machine learning probably can produce
| something akin to a brain, but LLMs are not really it
| even if they use the digital equivalent of a neuron. As
| much as I understand what I read about LLMs they really
| seem to be descendants of Markov chains. I think they are
| valuable and can go a long way, but LLMs themselves will
| not be "it". I think that we will get to a ceiling with
| them within 10 years if we will not think about something
| else. I think the ceiling can be made pretty high though.
|
| However most probably in 10 years we will all laugh how
| all of our predictions missed by a long shot.
| hospitalhusband wrote:
| LMM = Large Markov Model. I use that term because models
| like GPT-4 and friends are for all intents and purposes
| Markov chains with more data, more compute, some lossy
| compression, and a bit of nearest neighbor search. Next-
| word-engines.
|
| > why can't I say that your brain is nothing but a bunch
| of biological neurons trained using its input and
| intialized based on your genetics?
|
| Because we don't think one word at a time, and we don't
| restart from scratch for every subsequent word.
| esafak wrote:
| GPT is not Markovian; it has state.
| hospitalhusband wrote:
| Then it's a markov-like with state. Or as I've taken to
| calling them lately Markov+state. (I couldn't resist,
| sorry.)
|
| A truck towing a trailer isn't just a car because it
| pivots in the middle and has more wheels. It's
| fundamentals of operation are still closer to a car or
| truck without trailer than a bicycle.
|
| Humans can form thoughts and get to mostly correct
| answers even as a gut feeling, and the language to
| explain why/how need not even be present. We don't form
| thoughts one word at a time.
| sebzim4500 wrote:
| >Because we don't think one word at a time
|
| In what sense does an LLM think one word at a time that
| doesn't also apply to a person typing at a keyboard? I'm
| typing one word at a time right now, I assume you aren't
| about to declare me a markov chain. When I read my brain
| presumably ingests one word at a time (not sure if it's
| one exactly, but it can't be much more than one). It is
| of course true that I have some notion of what I'm going
| to say before I right the first word, but seemingly so
| does an LLM.
|
| If it was truly thinking one word at a time, it wouldn't
| be able to consistently use 'an' vs 'a' correctly, for
| example.
|
| >we don't restart from scratch for every subsequent word.
|
| LLMs don't restart from scratch for every word, via the
| attention heads they can look back through the entire
| context. Otherwise the memory required for inference
| wouldn't scale with the context length.
| hospitalhusband wrote:
| > In what sense does an LLM think one word at a time that
| doesn't also apply to a person typing at a keyboard?
|
| Because you already have the thought formed before you
| started typing.
|
| > When I read my brain presumably ingests one word at a
| time (not sure if it's one exactly, but it can't be much
| more than one)
|
| And these models ingest many vectors at once, up to the
| context length. Your brain is also recursive, and
| regularly goes backwards to rescan earlier words as
| necessary.
|
| Seems to me it's fundamentally inverted from how we
| operate, both input and output.
| sebzim4500 wrote:
| >Because you already have the thought formed before you
| started typing.
|
| Can you prove that GPT-4 doesn't? Clearly there is a
| sense in which thinks more than one word ahead, since as
| I mentioned above it would not otherwise be able to use
| 'a' vs 'an' correctly.
|
| As far as I am aware, exactly to what extent these models
| have determined what tokens will be generated before they
| produce anything is an open question in mechanistic
| interpratability research. I would be very interested if
| you knew of some work that answers this question
| empirically.
| [deleted]
| burnished wrote:
| Are you intentionally echoing that scene from starship
| troopers where the pundit makes the same claim about 'smart
| bugs'?
| hospitalhusband wrote:
| I don't base my ideas on what constitutes humanity,
| intelligence, or sentience on hundred million dollar
| fiction.
| burnished wrote:
| Alright then you should also consider not basing it off
| of whether you find the concept insulting. It doesn't
| seem to be the strongest rebuttal available.
| heyitsguay wrote:
| And moreover, this is more than a philosophical difference. As
| our understandings of AI and neuroscience improve we can now
| talk about "world models" as something almost tangible and
| constructible. Complex animal nervous systems build and use
| embodied world models to interpret the senses and allow for
| tractable future prediction and action generation. Yann
| Lecunn's paper on AGI from August 2022 gets into defining
| computational equivalents, and multimodal AI models like PaLM-E
| are taking first steps toward explicitly tying LLM-style text
| prediction with collections of perceptual observations about
| the environment.
|
| So it's not that artificial systems cannot, as a category, have
| models of their environment and the actions they can take
| within it, this "cargo cult AI" concept comes up when people
| jump the gun and see those capabilities in much simpler
| systems, even including ChatGPT-4. And it's disheartening to
| see narrative on the subject driven more and more by people who
| have not taken the time to learn about the subject matter, for
| all the interest they seem to show in talking about it.
| sharemywin wrote:
| I think with reinforcement learning and adversarial play you
| don't need it to understand for it to have unintended
| consequences and "break out of it's cage"
|
| Surprising behaviors We've shown that agents can learn
| sophisticated tool use in a high fidelity physics simulator;
| however, there were many lessons learned along the way to this
| result. Building environments is not easy and it is quite often
| the case that agents find a way to exploit the environment you
| build or the physics engine in an unintended way.
|
| Emergent tool use from multi-agent interaction
| https://openai.com/research/emergent-tool-use
|
| Here's an example where using a LLM enhanced a reinforcement
| algo performance.
|
| https://arxiv.org/abs/2302.06692
| intalentive wrote:
| > What makes human intelligence different from today's AI is the
| ability to ask why, reason from first principles, and create
| experiments and models for testing hypotheses.
|
| Too anthropocentric. Here is a video of cats "creating
| experiments" and "testing hypotheses":
| https://youtu.be/a_IA-8nQ4FY
|
| Michael Levin has showed that even single-celled organisms have
| apparently intelligent and goal-directed actions.
|
| Today's AI can't do that stuff either. If it could, we would have
| Rosie the Robot and C-3PO by now.
| luckydata wrote:
| they do that because survival, but I'm not sure we want to give
| that kind of motivation to our artificial intelligence
| algorithms
| igammarays wrote:
| LLM's are an incredibly useful tool which I plan to use
| extensively, but to think it is even in the same category as
| biological life is incredibly stupid.
|
| Man (and other animals) have Life -> Awareness -> Will -> Speech
| -> Power. ChatGPT only has Speech that is subject to our prompts.
| kerkeslager wrote:
| I'd argue that they don't even have speech in the same sense as
| we do. We choose words because they're connected to semantics
| we wish to convey, while LLMs choose words because they fit
| word patterns that exist in their training datasets.
___________________________________________________________________
(page generated 2023-05-18 23:02 UTC)