[HN Gopher] Getting 50% (SoTA) on Arc-AGI with GPT-4o
___________________________________________________________________
Getting 50% (SoTA) on Arc-AGI with GPT-4o
Author : tomduncalf
Score : 374 points
Date : 2024-06-17 21:51 UTC (1 days ago)
(HTM) web link (redwoodresearch.substack.com)
(TXT) w3m dump (redwoodresearch.substack.com)
| traject_ wrote:
| We don't actually know if it is SOTA, the previous SOTA solution
| also got around the same on the evaluation set.
| cma wrote:
| Yeah and GPT4o was potentially trained on this test set and if
| the tried to hold it out it was still likely trained on
| discussions of the problems.
| extr wrote:
| Very cool. When GPT-4 first came out I tried some very naive
| approaches using JSON representations on the puzzles [0], [1].
| GPT-4 did "okay", but in some cases it felt like it was falling
| for the classic LLM issue of saying all the right things but then
| then failing to grasp some critical bit of logic and missing the
| solution entirely.
|
| At the time I noticed that many of the ARC problems rely on
| visual-spatial priors that are "obvious" when viewing the grids,
| but become less so when transmuted to some other representation.
| Many of them rely on some kind of symmetry, counting, or the very
| human bias to assume a velocity or continued movement when seeing
| particular patterns.
|
| I had always thought maybe multimodality was key: the model needs
| to have similar priors around grounded physical spaces and
| movement to be able to do well. I'm not sure the OP really
| fleshes this line of thinking out, brute forcing python solutions
| is a very "non human" approach.
|
| [0] https://x.com/eatpraydiehard/status/1632671307254099968
|
| [1] https://x.com/eatpraydiehard/status/1632683214329479169
| refulgentis wrote:
| > brute forcing python solutions is a very "non human"
| approach.
|
| ARC-AGI has odd features that leave me flummoxed by the naming
| and the attendant prize money and hype.
|
| It is one singular task and frankly I _strongly_ suspect
| someone could beat it within 30 days[1], in an unsatisfying
| way, as you note.
|
| There's so much alpha that can be pieced together from here,
| ex. the last couple Google papers use the 1M context to do
| *500-shot*, i.e. 500 question answer examples. IIRC most recent
| showed raising travelling-salesman problem solve rate from 3 to
| 35%.
|
| [1] I pre-registered this via a Twitter post, about 48 hours
| ago, i.e. before this result was announced.
| elicksaur wrote:
| The private test set has been available to crack for almost
| four years now. There was also a monetary prize competition
| run last year.
|
| In your opinion, what has changed that would accelerate a
| solution to the next 30 days?
| refulgentis wrote:
| Prize money meant people would more cleverly strain the
| rule that "the private test set stays private, no GPT4o,
| Claude etc.", as shown by the TFA.
|
| This sort of idea would then be shared openly on new sites,
| creating more attempts. Fallout I did not anticipate was
| getting widespread attentional on general tech news sites,
| and then getting public comment from a prize co-founder
| confirming it was acceptable.
| elicksaur wrote:
| It seems like you don't understand the rules of the
| competition. Entries don't have access to the internet.
| The OP acknowledges in their post that this is not
| eligible for the prize. The HN comment from the prize co-
| founder specifically says the OP's claims haven't been
| scrutinized. (implicit: they won't be for the prize set
| unless the OP submits with an open LLM implementation)
|
| There is a plan for a "public" leaderboard, but it
| currently has no entries, so we don't actually know what
| the SOTA for the unrestrained version is. [1]
|
| The general idea - test time augmentation - is what the
| current private set SOTA uses. [2] Generating more
| examples via transforming the samples is not a new idea.
|
| Really, it seems like all the publicity has just gotten a
| bunch of armchair software architects coming up with 1-4
| year-old ideas thinking they are geniuses.
|
| [1] https://arcprize.org/leaderboard
|
| [2] https://lab42.global/community-interview-jack-cole/
| refulgentis wrote:
| > It seems like you don't understand the rules of the
| competition.
|
| I don't think you "don't understand" anything :) I'd ask
| you, politely, to consider that when you're replying to
| other people in the future.
|
| Better to bring to interactions the prior that your
| interlocutor is a presumably intelligent individual who
| can have a different interpretation of the same facts,
| than decide they just don't get it. The second is a quite
| lonely path.
|
| > Entries don't have access to the internet.
|
| Correct. Per TFA, cofounder, Chollet, then me: this _is
| an offline solution_ : the solution is the Python program
| found by an LLM.
|
| > The HN comment from the prize co-founder specifically
| says the OP's claims haven't been scrutinized.
|
| Objection: relevancy? Is your claim here that it might be
| false so we shouldn't be discussing it at all?
|
| > (implicit: they won't be for the prize set unless the
| OP submits with an open LLM implementation)
|
| I don't know what this means, "open LLM implementation"
| is either a term of art I don't recognize, or a
| misunderstanding of the situation.
|
| I do assume you read the article, so I'm not trying to
| talk down to you, but to clarify:
|
| The _solution is the Python program_ , not the LLM
| prompts that iterated on a Python program. A common
| thread that would describe the confusing experience of
| reading your comment phrased aggressively and disputing
| everything up until you agree with me: your observations
| assume I assume the solution requires a cloud-based LLM
| to run. As noted above, it doesn't, which is also the
| thrust of my comment: they found a way to skirt what I
| thought the rules are, and the co-founder and Chollett
| have embraced it, publicly.
|
| > There is a plan for a "public" leaderboard, but it
| currently has no entries, so we don't actually know what
| the SOTA for the unrestrained version is. [1]
|
| This was false before you posted, when I checked this
| morning, and it was false as early as 4 days ago, June
| 14th, we can confirm via archive.is. (prefix the URL you
| provided with archive.is/ to check for yourself)
|
| > The general idea - test time augmentation - is what the
| current private set SOTA uses. [2] Generating more
| examples via transforming the samples is not a new idea.
|
| Did anyone claim it was?
|
| > Really, it seems like all the publicity has just gotten
| a bunch of armchair software architects coming up with
| 1-4 year-old ideas thinking they are geniuses.
|
| I don't know what this means other than you're upset, but
| yes, sounds like both you and I agree that having an LLM
| generate Python programs isn't quite what we'd thought
| would be an AGI solution in the eyes of Chollet.
|
| Alas, here we are.
| nl wrote:
| (Not the OP)
|
| >> (implicit: they won't be for the prize set unless the
| OP submits with an open LLM implementation)
|
| > The solution is the Python program, not the LLM prompts
| that iterated on a Python program. A common thread that
| would describe the confusing experience of reading your
| comment phrased aggressively and disputing everything up
| until you agree with me: your observations assume I
| assume the solution requires a cloud-based LLM to run. As
| noted above, it doesn't, which is also the thrust of my
| comment: they found a way to skirt what I thought the
| rules are, and the co-founder and Chollett have embraced
| it, publicly.
|
| I think the implication is that solutions that use an LLM
| via an API won't be eligible (the "no internet" rule).
|
| This seems obvious to solve: can use GPT4 to generate
| catalogs in advance and a lesser, local LLM with good
| code abilities to select them.
|
| I don't see why this skirts any rules you think were
| implied and I'm puzzled why you think it does.
|
| > sounds like both you and I agree that having an LLM
| generate Python programs isn't quite what we'd thought
| would be an AGI solution in the eyes of Chollet.
|
| > Alas, here we are.
|
| Chollet noted that program synthesis was a promising
| approach, so it's not surprising to me that a program
| synthesis approach that also uses an LLM is effective.
| elicksaur wrote:
| From the leaderboard link (and on the archive version):
|
| >ARC-AGI-Pub is a secondary leaderboard (in beta)
| measuring the public evaluation set. ... The public
| evaluation set imposes no limitations on internet access
| or compute. At this time, ARG-AGI-Pub is not part of ARC
| Prize 2024 (eg. no prizes are associated with this
| leaderboard).
|
| And, all the entries at time of writing and in the
| archive link say "You?...". "ARC-AGI 2024 HIGH SCORES"
| which does have entries is on the private test set.
|
| >I don't think you "don't understand" anything :)
|
| _I genuinely don't understand_ if we are viewing the
| same websites.
| refulgentis wrote:
| > I genuinely don't understand if we are viewing the same
| websites.
|
| We are! I missed the nuance on you're looking for a
| _public_ leaderboard on the _private_ test set. I do see
| it now, but I 'm still confused as to how that's relevant
| here.
| YeGoblynQueenne wrote:
| I wish this comment was less confrontational because
| there's useful information in it and several points I
| agree with.
| elicksaur wrote:
| Hey! Genuinely, thank you for the feedback!
| nl wrote:
| I don't think this is "non-satisfying" at all.
|
| Program synthesis has been mentioned as a promising approach
| by Francois Chollet, and that's exactly what this is.
|
| The place I find slightly unsatisfying is this:
|
| > Sample vast, vast numbers of completions (~5,000 per
| problem) from GPT-4o.
|
| > Take the most promising 12 completions for each problem,
| and then try to fix each by showing GPT-4o what this program
| actually outputs on the examples, and then asking GPT-4o to
| revise the code to make it correct. We sample ~3,000
| completions that attempt to fix per problem in total across
| these 12 starting implementations.
|
| I'd been tossing around a MCTS idea similar to AlphaGo, based
| on the idea that the end transformation is a series of sub-
| transformations. I feel like this could work well alongside
| the GPT-4o completion catalog. (This isn't an original
| observation or anything)
| bubblyworld wrote:
| Classic, I've been doing the same, writing an alphazero for
| the transformation part. What seems _much_ harder is
| picking a decent set of transformations/concepts to work
| with, or more generally automating that process. Maybe
| you're right that LLMs could help there!
| luke-stanley wrote:
| Reminds me of NVIDIA Eureka: https://github.com/eureka-
| research/Eureka
| bubblyworld wrote:
| Very nice! Thanks for the link, that's great inspiration.
| nl wrote:
| Great link, thanks.
| YeGoblynQueenne wrote:
| >> Program synthesis has been mentioned as a promising
| approach by Francois Chollet, and that's exactly what this
| is.
|
| To be precise, "this" -a bog-standard generate-and-test
| approach- is the dumbest possible way to do program
| synthesis. It's like sorting lists with bogosort and a very
| big computer.
|
| It's _exactly_ like bogosort: generate permutations and
| test. Except of course the system that generates
| permutations costs a few millions(?).
| bubblyworld wrote:
| Bogosort is driven by 0 heuristics - just shuffle and
| play. Using an LLM as a high-level prior over your search
| is very different, and the author had to do a lot of
| problem-specific tuning to make it work well.
| YeGoblynQueenne wrote:
| But he tuned the "test" side of the generate-and-test
| loop, not the "generate" side. The "generate" side
| remains a big permutation generator that is also very
| hard to control. The current highest-ranked system on the
| private test set of the ARC-AGI (at 34%) is another LLM
| fine-tuned on manually created examples of ARC tasks, so
| that would indeed be messing with the generator part of
| the loop. I'm guessing performance will jump when someone
| puts the two together.
|
| A heuristic btw, is something completely different than
| fine tuning, or filtering. Heuristic search is the
| closest thing we have to an approximation of the kind of
| goal-driven behaviour we see in animal intelligence.
|
| I think you could argue that gradient optimisation or any
| kind of optimisation of some kind of objective function
| is the same (Rich Sutton has a paper titled "Reward is
| all you need"). I'm not sure where I stand with that.
| YeGoblynQueenne wrote:
| >> GPT-4 did "okay", but in some cases it felt like it was
| falling for the classic LLM issue of saying all the right
| things but then then failing to grasp some critical bit of
| logic and missing the solution entirely.
|
| It still is. It misses the solution so comprehensively that it
| needs an outer loop to figure out which one is the solution out
| of 8k programs GPT-4o generates.
| ealexhudson wrote:
| We don't really know what GPT-4 "is". I remember reading a
| number of relatively well-informed suggestions that there are
| a number of a models inside there, and the API being
| interacted with is some form of outer-loop around them.
|
| I don't think the location of the outer-loop or the design of
| it really makes much difference. There is no flock of birds
| without the individuals, the flock itself doesn't really
| exist as a tangible thing, but what arises out of the
| collective adjustments between all these individuals gives
| rise to a flock. Similarly, we may find groups of LLMs and
| various outer control loops give rise to an emergent
| phenomena much greater than the sum of their parts.
| YeGoblynQueenne wrote:
| >> We don't really know what GPT-4 "is".
|
| Yes, we do. It's a language model.
| greatpostman wrote:
| You know you're approaching AGI when creating benchmarks gets
| difficult. This is only just beginning
| elicksaur wrote:
| Alternatively, society has no common understanding of what AGI
| means.
| Workaccount2 wrote:
| Which is why the first "artificial" sentient beings will
| likely go through the wringer that humanity historically put
| other "sub-human" beings through.
| Slyfox33 wrote:
| Benchmarks being difficult to create has no connection to
| something being agi.
| rgbrgb wrote:
| > 50% accuracy on the public test set for ARC-AGI by having
| GPT-4o
|
| Isn't the public test set public on github and therefore GPT-4o
| trained on it?
| bongodongobob wrote:
| I keep seeing this comment all over the place. Just because
| something exists 1 time in the training data doesn't mean it
| can just regurgitate that. That's not how training works. An
| LLM is not a knowledge database.
| adroniser wrote:
| And yet it doesn't rule out that it can't. See new york times
| lawsuit
| bongodongobob wrote:
| From old pieces of articles that are quoted all over the
| internet? That's not surprising.
| ben_w wrote:
| That's still sufficient for both The Times and for it to
| be a potential problem in this case.
| spencerchubb wrote:
| It could exist many times. People can fork and clone the
| repo. People are likely to copy the examples and share them
| online.
| daemonologist wrote:
| In this case I don't think having seen the Arc set would help
| much in writing and selecting python scripts to solve the test
| cases. (Unless someone else has tried this approach before and
| _their_ results are in the training data.)
|
| It will be good to see the private set results though.
| cma wrote:
| Public discussions of solutions to the public test set will
| presumably have somewhat similar analogies and/or embeddings
| to aspects of the python programs that solve them.
| Truth_In_Lies wrote:
| Yes, someone has tried https://iprc-dip.github.io/DARC/
| bashfulpup wrote:
| I looked at the website and have no idea how Arc is supposed to
| be AGI.
|
| Can someone explain?
| TheDudeMan wrote:
| It is necessary but not sufficient.
|
| If you can't do ARC, you aren't general enough. But even if you
| can do ARC, you still might not be general enough.
| Workaccount2 wrote:
| It's also possible that you are an AGI and simply cannot pass
| ARC.
| TheDudeMan wrote:
| How so? If there is a task that humans can do but the AI
| cannot, I would not call it AGI. But that's just my
| definition.
| awwaiid wrote:
| Yeah but if my brother can't pass it, that doesn't mean
| he is NOT human.
| Jensson wrote:
| Could he pass it if he was educated to do the task from
| birth? Human level intelligence includes being able to be
| educated, the ML models we have done so far can't be
| educated so have to match the level of educated humans to
| compare.
|
| General intelligence as we know it requires ability to
| receive education.
| infgeoax wrote:
| Isn't chatGPT already proven to be smarter than many of
| us in many ways?
| TheDudeMan wrote:
| I said AGI. I did not say human.
| biophysboy wrote:
| Its not a test of AGI. It tests whether you possess innate
| human capacities: rudimentary arithmetic & geometry, etc. Most
| of the problems were created manually. The original paper
| states that they limited the test to innate human priors to
| make the scope well-defined.
| eigenvalue wrote:
| The Arc stuff just felt intuitively wrong as soon as I heard it.
| I don't find any of Chollet's critiques of LLMs to be convincing.
| It's almost as if he's being overly negative about them to make a
| point or something to push back against all the unbridled
| optimism. The problem is, the optimism really seems to be
| justified, and the rate of improvement of LLMs in the past 12
| months has been nothing short of astonishing.
|
| So it's not at all surprising to me to see Arc already being
| mostly solved using existing models, just with different
| prompting techniques and some tool usage. At some point, the
| naysayers about LLMs are going to have to confront the problem
| that, if they are right about LLMs not really
| thinking/understanding/being sentient, then a very large
| percentage of people living today are also not
| thinking/understanding/sentient!
| HarHarVeryFunny wrote:
| Actually the solution being discussed here is the one that
| Chollet mentioned in his interview with Dwarkesh, and only
| bolsters his case.
|
| The LLM isn't doing the reasoning here, it's just pattern
| matching the before/after diff and generating thousands of
| Python programs. The actual reasoning is done by an agentic
| like loop wrapped around the LLM, as described in the linked
| blog.
| awwaiid wrote:
| When you peer into the soul of the machine it delicately
| resolves to `while(1){...}`. All Hail The REPL.
| Smaug123 wrote:
| > a very large percentage of people living today are also not
| thinking/understanding/sentient
|
| This isn't _that_ big a bullet to bite
| (https://www.lesswrong.com/posts/4AHXDwcGab5PhKhHT/humans-
| who... comes from well before ChatGPT's launch), and I myself
| am inclined to bite it. System 1 alone does not a general
| intelligence make, although the article is extremely
| interesting in asking the question "is System 1 plus Python
| enough for a general intelligence?". But it's not a very
| relevant philosophical point, because Chollet's position is
| consistent with humans being obsoleted and/or driven extinct
| whether or not the LLMs are "general intelligences".
|
| His position is that training LLMs results in an ever-larger
| number of learned algorithms and no ability to construct _new_
| algorithms. This is consistent with the possibility that, after
| some threshold of size and training, the LLM has learned every
| algorithm it needs to supplant humans in (say) 99.9% of cases.
| (It would definitely be going out with a whimper rather than a
| bang, on that hypothesis, to be out-competed by something that
| _really is_ just a gigantic lookup table!)
| threeseed wrote:
| a) 50% result is not solving the problem. Especially when the
| implementation is brute forcing the problem and is against the
| spirit of ARC.
|
| b) He is not being overly negative of LLMs. In fact he believes
| they will play a role in any AGI system.
|
| c) OpenAI CTO has publicly said that ChatGPT 5 will not be
| significantly better than existing models. So the rate of
| improvements you believe in simply doesn't match reality.
| janalsncm wrote:
| For the record, a lot of problems might turn out to be like
| this, where we figure out a brute force approach that stands
| in for human creativity.
| hackerlight wrote:
| Skeptical about (c), source please. She did say they don't
| have anything much better than GPT-4o currently, but GPT-5
| likely only started training recently.
| traject_ wrote:
| > It's almost as if he's being overly negative about them to
| make a point or something to push back against all the
| unbridled optimism.
|
| I don't think it is like that but rather Chollet wants to see
| stronger neuroplasticity in these models. I think there is a
| divide between the effectiveness of existing AI models versus
| their ability to be autonomous, robust and consistently learn
| from unanticipated problems.
|
| My guess is Chollet wants to see something more similar to
| biological organisms especially mammals or birds in their level
| of autonomous nature. I think people underestimate the degree
| of novel problems birds and mammals alone face in just simply
| navigating their environment and it is the comparison here that
| LLMs, for now at least, seem lacking.
|
| So when he says LLMs are not sentient, he's asking to consider
| the novel problems animals let alone humans have to face in
| navigating their environment. This is especially apparent in
| young children but declines as we age and gain experience/lose
| a sense of novelty.
| infgeoax wrote:
| Agree. When I first saw ARC, my reaction was this could
| possibly be the kind of problem that gives us evolutionary
| pressure.
| adroniser wrote:
| I don't see how the point about the typical human is relevant.
| Either you can reason or you can't, the ARC test is supposed to
| be an objective way to measure this. Clearly a vanilla LLM
| currently cannot do this, and somehow an expert crafting a
| super-specific prompt is supposed to be impressive.
| eigenvalue wrote:
| The point is that if you have some test of whether an AI is
| intelligent that the vast majority of living humans would
| fail or do worse on than gpt4-o (let alone future LLMs) then
| it's not a very persuasive argument.
| TacticalCoder wrote:
| > I don't find any of Chollet's critiques of LLMs to be
| convincing. It's almost as if he's being overly negative about
| them to make a point or something to push back against all the
| unbridled optimism.
|
| Chollet published his paper _On the measure of intelligence_ in
| 2019. In Internet time that is a lifetime before the LLM hype
| started.
| refulgentis wrote:
| Einstein, infamously, couldn't really make much progress with
| quantum physics, even though he invented the precursors (ex.
| Brownian motion). Your world model is hard to update.
| imperfect_light wrote:
| A bit of a stretch given that Chollet is a researcher in
| deep learning and transformers and his criticism is that
| memorization (training LLMs on lots and lots of problems)
| doesn't equate to AGI.
| refulgentis wrote:
| > A bit of a stretch
|
| Is that true?
|
| C.f. what we're discussing
|
| He's actively encouraging using _LLMs_ to solve his
| benchmark, called ARC _AGI_.
|
| 8 hours ago, from Chollet, re: TFA
|
| "The best solution to fight combinatorial explosion is to
| leverage intuition over the structure of program space,
| provided by a deep learning model. For instance, _you can
| use a LLM to sample a program_... "
|
| Source: https://x.com/fchollet/status/1802801425514410275
| imperfect_light wrote:
| The stretch was in reference to comparing Chollet to
| Einstein. Chollet clearly understands LLMs (and
| transformers and deep learning), he simply doesn't
| believe they are sufficient for AGI.
| refulgentis wrote:
| I don't know what you mean, it's a straightforward
| analogy, but yes, that's right, except for the part where
| he's heralding this news by telling people the LLM is an
| underexplored solution space for a possible solution to
| his AGI benchmark he made to disprove LLMs are AGI.
|
| I don't mean to offend, but to be really straightforward:
| he's the one saying it's possible they might be AGI now.
| I'm as flummoxed as you, but I think its hiding the ball
| to file it under "he doesn't mean what he's saying,
| because he doesn't believe LLMs can ever be AGI." The
| only steelman for that is playing at: AGI-my-benchmark,
| which I say is for AGI, is not the AGI I mean
| imperfect_light wrote:
| You're reading a whole lot into a tweet, in his interview
| with Dwarkesh Patel he says, about 20 different times,
| that scaling LLMs (as they are currently conceived) won't
| lead to AGI.
| anoncareer0212 wrote:
| You keep changing topics so I don't get it either, I can
| attest it's not a fringe view that the situation is
| interesting, seen it discussed several times today by
| unrelated people.
| imperfect_light wrote:
| He's said it pretty clearly, an LLM could be part of the
| solution in combination with program synthesis, but an
| LLM alone won't achieve AGI.
| infgeoax wrote:
| But it's his EPR paper inspired the Bell's inequality and
| pushed the field further. Yes he was wrong about how
| reality works, but still he asked the right question.
| gwern wrote:
| From Chollet's perspective, the LLM hype started well before,
| with at least GPT-2 half a year before his paper, and he
| spent plenty of time mocking GPT-2 on Twitter before he came
| up with ARC as a rebuttal.
| modeless wrote:
| It's a very convincing rebuttal considering that GPT-3 and
| GPT-4 came out after ARC but made no significant progress
| on it. He seemingly had the single most accurate and
| verifiable prediction of anyone in the world (in 2019)
| about _exactly_ what type of tasks scaled LLMs would be bad
| at.
| imtringued wrote:
| Yeah I agree. We have reached the end of LLMs. LLMs are
| infallible and require no further improvement. Anyone who
| points out shortcomings of current architectures and training
| approaches should be ignored as a naysayer. Anyone who proposes
| a solution to perceived flaws is a crank trying to fix
| something that was never broken. Everyone knows humans are
| incapable of internal monologues or visualization and
| vocalisation. Humans don't actually move their lips to speak to
| produce a sound that can be interpreted by a speaker of the
| same language, they produce universally understood tokens
| encoding objective reality and the fact that they use the local
| language is merely a habit that is hard to break out of.
| mrtranscendence wrote:
| Sometimes, when I'm undertaking the arduous work of assigning
| probabilities to everything I could possibly say next in a
| conversation, I wish that I weren't merely a stochastic
| autoregressive next-token generator. Them's the breaks,
| though.
| biophysboy wrote:
| I don't think he's as critical as you say. He just views LLMs
| as the product of intelligence rather than intelligence itself.
| LLM fans will say this is a false distinction, I guess.
|
| His definition of intelligence is interesting: something that
| can quickly achieve tasks with few priors or experience. I also
| think the idea of using human "Core Knowledge" priors is a
| clever way to make a test.
| lassoiat wrote:
| I am a chatGPT fan boy and have been quite impressed by 4o but
| I will really be impressed when it stops inventing aspects of
| python libraries that don't exists and instead just tells me it
| doesn't exist.
|
| It literally just did this for me 15 minutes ago. You can't
| talk about AGI when it is this easy to push it over the edge
| into something it doesn't know.
|
| Paper references have got better the last 12 months but just
| this week it made up both a book and paper for me that do not
| exist. The authors exist and they did not write what it said
| they did.
|
| It is very interesting if you ask "do you understand your
| responses?" sometimes it will say yes and sometimes it will so
| no not like a human understands.
|
| We should forget about AGI until it can at least say it doesn't
| know something. It is hardly a sign of intelligence in humans
| to make up answers to questions you don't know.
| motoxpro wrote:
| Every time you're wrong and you disagree with someone who is
| right you are inventing things that don't exist.
|
| Unless you're saying you have never held on to a wrong
| opinion that was at some point proven to be wrong?
| imperfect_light wrote:
| Did you listen to what Chollet said? How much of LLM
| improvements are due to enlarging the training sets to cover
| more problems and how much is due to any emergent properties?
| Lockal wrote:
| That's a big jump in generalization that bruteforcing 4 colors
| in 9x9 grids with 8000 programs has anything near to what
| sentient human can do.
|
| Back in the days similar generalization was used for Deep Blue
| chess computer. Computer won in 1997, but the AGI abyss is
| still as big.
| mikeknoop wrote:
| (ARC Prize co-founder here).
|
| Ryan's work is legitimately interesting and novel "LLM reasoning"
| research! The core idea:
|
| > get GPT-4o to generate around 8,000 python programs which
| attempt to implement the transformation, select a program which
| is right on all the examples (usually there are 3 examples), and
| then submit the output this function produces when applied to the
| additional test input(s)
|
| Roughly, he's implemented an outer loop and using 4o to sample
| reasoning traces/programs from training data and test. Hybrid DL
| + program synthesis approaches are solutions we'd love to see
| more of.
|
| A couple important notes:
|
| 1. this result is on the public eval set vs private set (ARC
| Prize $).
|
| 2. the current private set SOTA ~35% solution also performed ~50%
| on the public set. so this new result _might_ be SOTA but hasn 't
| been validated or scrutinized yet.
|
| All said, I do expect verified public set results to flow down to
| the private set over time. We'll be publishing all the SOTA
| scores and open source reproductions here once available:
| https://arcprize.org/leaderboard
|
| EDIT: also, congrats and kudos to Ryan for achieving this and
| putting the effort in to document and share his approach. we hope
| to inspire more frontier AI research sharing like this
| refibrillator wrote:
| Do you have any perspectives to share on Ryan's observation of
| a potential scaling law for these tasks and his comment that
| "ARC-AGI will be one benchmark among many that just gets solved
| by scale"?
| mikeknoop wrote:
| ARC isn't perfect and I hope ARC is not the last AGI
| benchmark. I've spoken with a few other benchmark creators
| looking to emulate ARC's novelty in other domains, so I think
| we'll see more. The evolution of AGI benchmarks likely needs
| to evolve alongside the tech -- humans have to design these
| tasks today to ensure novelty but should expect that to
| shift.
|
| One core idea we've been advocating with ARC is that pure LLM
| scaling (parameters...) is insufficient to achieve AGI.
| Something new is needed. And OPs approach using a novel outer
| loop is one cool demonstration of this.
| Nimitz14 wrote:
| Ah that's an important detail about public v private. Makes it
| a nice result but nearly as impressive as initially stated.
| hackerlight wrote:
| Reminds me of the AlphaCode approach.
|
| Why do you say it's sampling programs from "training data"?
| With that choice of words, you're rhetorically assuming the
| conclusion.
|
| If he only sampled 20 programs, instead of 8000, will we still
| say the programs came from "training data", or will we say it's
| genuine OOD generalization? At what point do we attribute the
| intelligence to the LLM itself instead of the outer loop?
|
| This isn't meant to be facetious. Because clearly, if the N
| programs sampled is very large, it's easy to get the right
| solution with little intelligence by relying on luck. But as N
| gets small the LLM has to be intelligent and capable of OOD
| generalization, assuming the benchmark is good.
| refreshingdrink wrote:
| Also worth nothing that Ryan mentions
|
| > In addition to iterating on the training set, I also did a
| small amount of iteration on a 100 problem subset of the public
| test set
|
| and
|
| > it's unfortunate that these sets aren't IID: it makes
| iteration harder and more confusing
|
| It's not unfortunate: generalizing beyond the training
| distribution is a crucial part of intelligence that ARC is
| trying to measure! Among other reasons, developing with test-
| set data is a bad practice in ML because it hides the
| difficulty this challenge. Even worse, writing about a bunch of
| tricks that help results on this subset is extending the test-
| set leakage the blog post's readers. This is why I'm glad the
| ARC Prize has a truly hidden test set
| rfoo wrote:
| ... and we know that if we really want to nail it we'd better
| just pay someone else to create 1,000,000 more harder
| problems for training (without looking at any in test set, of
| course). i.e. make the training set distribution similar to
| test set again.
|
| Because the thing we have now is data-hungry. Your brain is
| pre-trained on other similar challenges as well. What's the
| point of requiring it to "generalize beyond the training
| distribution" with so few samples?
|
| Really, I thought LLMs ended this "can we pretrain on in-
| house prepared private data for ILSVRC" flame war already.
| YeGoblynQueenne wrote:
| The problem with that it is we know approaches that can
| generalise very well from very few examples, even one
| example, without any kind of pretraining, That requires a
| good background theory of the target domain (a "world
| model" in more modern parlance), and we don't know how to
| automatically generate that kind of theory; only human
| minds can do it, for now. But given such a theory the
| number of examples needed can be as few as 1. Clearly, if
| you can learn from one example, but find yourself using
| thousands, you've taken a wrong turn somewhere.
|
| The concern with the data-hungry approach to machine
| learning, that at least some of us have, is that it has
| given up on the effort to figure out how to learn good
| background theories and turned instead to getting the best
| performance possible in the dumbest possible way, relying
| on the largest available amount of examples and compute.
| That's a trend against everything else in computer science
| (and even animal intelligence) where the effort is to make
| everything smaller, cheaper, faster, smarter: it's putting
| all the eggs in the basket of making it big, slow and dumb,
| and hoping that this will somehow solve... intelligence. A
| very obvious contradiction.
|
| Suppose we lived in a world that didn't have a theory of
| computational complexity and didn't know that some programs
| are more expensive to run than others. Would it be the case
| in that world, that computer scientists competed in solving
| ever larger instances of the Traveling Salesperson Problem,
| using ever larger computers, without even trying to find
| good heuristics exploiting the structure of the problem and
| simply trying to out-brute-force each other? That world
| would look a lot like where we are now with statistical
| machine learning: a pell-mell approach to throwing all
| resources at a problem that we just don't know how to
| solve, and don't even know if we can solve.
| advael wrote:
| The formalism that data-driven machine learning leans on
| is empirical tuning of stochastic search to drive
| approximation of functions, and despite what Silicon
| Valley would have you believe, most of the significant
| advances have been in creating useful meta-structures for
| modeling certain kinds of problems (e.g. convolution for
| efficiently processing transformations that care about
| local structure across dimensions of data, or qkv
| attention for keeping throughlines of non-local
| correspondences intact through a long sequence). Neural
| networks as a flavor of empirical function approximation
| happened to scale well, and then a bunch of people who
| saw how much this scale improved the models' capabilities
| but couldn't be bothered to understand the structural
| component concluded that scale somehow magically gets you
| to every unsolved problem being solved. It's also
| convenient for business types that if you buy this
| premise, any unicorn they want to promise is just a
| matter of throwing obscene amounts of resources at the
| problem (through their company of course)
|
| I think probably the general idea of dynamic structures
| that are versatile in their ability to approximate
| functional models is at least a solid hypothesis for how
| some biological intelligence works at some level (I think
| maybe the "fluid/crystallized" intelligence distinction
| some psychology uses is informative here - a strong world
| model probably informs a lot of quick acquisition of
| relationships, but most intelligent systems clearly
| posess strong feedback mechanisms for capturing new
| models), though I definitely agree that a focus on how
| best to throw a ton of scale at these models doesn't seem
| like a fruitful path for actionably learning how to build
| or analyze intelligent systems in the way we usually
| think about, nor is it, well, sustainable. Moore's law
| appeals to business people because buying more
| computronium feels more like a predictable input-output
| relationship to put capital into, but even if we're just
| talking about raw computation speed advances in
| algorithms tend to dwarf advances in computing power in
| the long run. I think the same will hold true in AGI
| YeGoblynQueenne wrote:
| Yeah, very good points. To be fair there are people who
| have argued the big data side who have clearly solid
| knowledge of AI and are not just SV suits, for example I
| remember Yann LeCun in a debate with Christopher Manning,
| where Manning was arguing for the importance of
| "structure" and LeCun was arguing against it. Or see the
| "Bitter Lesson", mentioned in a parent comment. That may
| have become a total shibboleth of the Silicon bros but
| Rich Sutton, who wrote the eponymous article, is the guy
| who wrote the book on Reinforcement Learning (literally).
| And then Rodney Brooks' replied with his "Better Lesson"
| (https://rodneybrooks.com/a-better-lesson/). So there's a
| lot of debate in this and I don't reckon we'll have a
| consensus soon. It should be clear which side I'm on- I
| work with firmly model-based AI ("planning is the model-
| based approach to autonomous behaviour" has become my
| shibboleth - see Bonnet and Geffner's book on planning:
| https://link.springer.com/book/10.1007/978-3-031-01564-9)
| so maybe it's a _deformation professionelle_. And even
| LCun 's recent plans for JEPA are very consciously model-
| based, except he wants to learn his models from data;
| which is not a bad idea I suppose.
| advael wrote:
| I've commented here before that I find myself really
| conflicted on LeCunn's public statements. I think it's
| really hard to reconcile the fact that he's undeniably a
| world-leading expert with the fact that he does work for
| and represent a tech company in a big way, which means
| that it's both hard to tell when what he says, especially
| publicly, is filtered through that lens, either
| explicitly or just via cultural osmosis. I know some
| people still in academia (e.g. "Bitter Lesson") are
| following suit but given how much of this field has been
| scooped up by large tech firms, this necessarily means
| that what we get out of research from those firms is
| partially filtered through them. Like it sounds like
| you're in CS/AI academia so I'm sure you're familiar with
| the distorting effect this brain drain has had on the
| field. Research out of places like FAIR or deepmind or
| OpenAI (arguably they were different until about 2019 or
| so? Hard to say how much of that was ever true
| unfortunately) are being done and published by world-
| leading experts hired by these companies and obviously
| this research has continued to be crucial to the field,
| but the fact that it's in industry means there's
| obviously controls on what they can publish, and the
| culture of an institution like Facebook is definitely
| going to have some different effects on priorities than
| that of most universities, and so while we can all
| collectively _try_ to take it all with a grain of salt in
| some way, there is no way to be careful enough to avoid
| tribal knowledge in the field being heavily influenced by
| the cultures and priorities of these organizations.
|
| But even if this kind of thinking is totally organic, I
| think it could arise from the delayed nature of the
| results of data-driven methods. Often a major structural
| breakthrough for a data-driven approach drastically
| predates the most obviously impactful results from that
| breakthrough, because the result impressive enough to
| draw people's attention comes from throwing lots of data
| and compute at the breakthrough. The people who got the
| impressive result might not even be the same team as the
| one that invented the structure they're relying on, and
| it's really easy to get the impression that what changed
| the game was the scale alone, I imagine even if you're on
| one of those research teams. I've been really impressed
| by some of the lines of research that show that you can
| often distill some of these results to not rely so
| heavily on massive datasets and enormous parallel
| training runs, and think we should properly view results
| that come from these to be demonstrations of the power of
| the underlying structural insights rather than new
| results. But I think this clashes with the organizational
| priorities of large tech firms, which often view scale as
| a moat, and thus are motivated to emphasize the need for
| it
| barfbagginus wrote:
| The recent result shows SOTA progress from something as
| goofy as generating 5000 python programs until 0.06% of
| them pass the unit tests. We can imagine our own brains
| having a thousand random subconscious pre thoughts before
| our consciously registered though is chosen and amplified
| out of the hallucinatory subconscious noise. We're still
| at a point where we're making surprising progress from
| simple feedback loops, external tools and checkers,
| retries, backtracking, and other bells and whistles to
| the LLM model. Some of these even look like world models.
|
| So maybe we can cure LLMs of the hallucinatory leprosy
| just by bathing them about 333 times in the mundane
| Jordan river of incremental bolt ons and modifications to
| formulas.
|
| You should be able to think of the LLM as a random
| hallucination generator then ask yourself "how do I wire
| ten thousand random hallucination generators together
| into a brain?" It's almost certain that there's an
| answer... And it's almost certain that the answer is even
| going to be very simple in hindsight. Why? Because llms
| are already more versatile than the most basic components
| of the brain and we have not yet integrated them in the
| scale that components are integrated in the brain.
|
| It's very likely that this is what our brains do at the
| component level - we run a bunch of feedback coupled
| hallucination generators that, when we're healthy,
| generates a balanced and generalizing consciousness - a
| persistent, reality coupled hallucinatory experience that
| we sense and interpret and work within as the world
| model. That just emerges from a network of self
| correcting natural hallucinators. For evidence, consider
| work in Cortical Columns and the Thousand brains theory.
| This suggests our brains have about a million Cortical
| Columns. Each loads up random inaccurate models of the
| world... And when we do integration and error correction
| over that, we get a high level conscious overlay. Sounds
| like what the author of the currently discussed SOTA did,
| but with far more sophistication. If the simplest most
| obvious approach to jamming 5,000 llms together into a
| brain gives us some mileage, then it's likely that more
| reasoned and intelligent approach could get these things
| doing feats like the fundamentally error prone components
| of our own brains can do when working together.
|
| So I see absolutely no reason we couldn't build an
| analogy of that with llms as the base hallucinator. They
| are versatile and accurate enough. We could also use
| online training llms and working memory buffers as the
| base components of a Jepa model.
|
| It's pretty easy to imagine that a society of 5000 gpt4
| hallucinators could, with the right self administered
| balances and utilities, find the right answers. That's
| what the author did to win the 50%.
|
| Therefore I propose that for the current generation it's
| okay to just mash a bunch of hallucinators together and
| whip them into the truth. We should be able to do it
| because our brains have to be able to do it. And if
| you're really smart, you will find a very efficient
| mathematical decomposition... Or a totally new model. But
| for every current LLM inability, it's likely to turn out
| that sequence of simple modifications can solve it. Will
| probably accrue a large number of such modifications
| before someone comes along and thinks of an all-new model
| then does way better, perhaps taking inspirations from
| the proposed solutions, or perhaps exploring the negative
| space around those solutions.
| yccs27 wrote:
| Sadly, right now the "throw lots of compute at it in the
| dumbest possible way" models work, and the "learn good
| background theories" approaches have gone nowhere. It's
| Rich Sutton's Bitter Lesson and a lot of us aren't ready
| to accept it.
|
| http://www.incompleteideas.net/IncIdeas/BitterLesson.html
| lesuorac wrote:
| > that ``brute force" search may have won this time, but
| it was not a general strategy, and anyway it was not how
| people played chess. These researchers wanted methods
| based on human input to win and were disappointed when
| they did not.
|
| Mostly tangential to the article but I never really like
| this argument. Like you're playing a game a specific way
| and somebody else comes in with a new approach and mops
| the floor with you and you're going to tell me "they
| played wrong"? Like no, you were playing wrong the whole
| time.
| entropicdrifter wrote:
| Yeah, people get salty when their preconceptions are
| shattered, especially when they've invested a lot of
| time/energy in thinking based on the idea that they were
| sound.
|
| It goes beyond simple sunk cost and into the realm of
| reality slapping them with a harsh "humans aren't
| special, grow up", which I think is especially bitter for
| people who aren't already absurdists or nihilists.
| advael wrote:
| You seem to misunderstand why generalization is important
| for making claims about intelligent systems. To illustrate
| this, we could really easily design a system that encodes
| all the test set questions and their answers, puts them in
| an enormous hash table, and looks up the correct answer to
| each challenge when presented with it. This could probably
| score 100% on ARC if given the entire test set. Would you
| call this AGI? What if I put it through a transformer as a
| hashing function?
|
| The mainstream attention LLMs have garnered has added a
| bunch of noise to the way we talk about machine learning
| systems, and unfortunately the companies releasing them are
| partially to blame for this. That doesn't mean we should
| change the definition of success for various benchmarks to
| better suit lay misunderstandings of how this all works
| rfoo wrote:
| First, LLMs are not AGI. Never will be. Can we talk now?
|
| > if given the entire test set.
|
| I don't want the entire test set. Or any single one in
| the test set.
|
| The problem here is ARC challenge deliberately give a
| training set with different _distribution_ than both the
| public and the private test set. It 's like having only
| 1+1=2, 3+5=8, 9+9=18 in training set and then 1+9=10,
| 5*5=25, 16/2=8, (0!+0!+0!+0!)!=24 in test set.
|
| I can see the argument of "giving the easy problems as
| demonstration of rules and then with 'intelligence' [1]
| you should be able to get harder ones (i.e. a different
| distribution)", but I don't believe it's a good way to
| benchmark current methods, mainly because there are
| shortcuts. Like I can teach my kids how factorial works
| and ! means factorial, instead of teaching them how
| addition works only and make them figure out how
| multiplication, division and factorial works and what's
| the notation.
|
| [1] Whatever that means.
| advael wrote:
| Okay I admit I'm confused and think I probably missed a
| crucial thing here. You're saying the publicly available
| problem set isn't indicative of the distribution of the
| test set? If so, I can see why you object to that. Still,
| it's potentially possible that the test's intention is to
| demonstrate something like progressive integration of
| compositionality given an abstract model. A lot of
| machine learning systems can do well as long as they've
| seen an example similar to the problem they've been
| presented, but can't do things like respond to a
| situation that presents them with a novel composition of
| two abstractions they seem to have already learned in the
| way a human can trivially.
|
| Like only having [1+1=2, 4+5=9, 2+10=12] in the training
| set and [2*5=10, 3/4=.75, 2^8=256] in the test set would
| be bad, but something like [1+1=2, 3+4*2=11, 5*3=15,
| 2*7=14, 1+3/5=1.8, 3^3=27] vs [2+4*3=14, 3+3^2+4=16,
| 2*3/4+2^3/2^4=2] might not be, depending on what they're
| trying to test
|
| Compositionality of information, especially of
| abstractions (like rules or models of a phenomenon), is a
| key criterion in a lot of people's attempts to
| operationally define "intelligence" (which I agree is
| overall a nebulous and overloaded concept, but if we're
| going to make claims about it we need at least a working
| definition for any particular test we're doing) I could
| see that meaning that the test set problems need to be
| "harder" in the sense that presenting compositions of
| rules in training doesn't preclude memorizing the
| combinations. But this is just a guess, I'm not involved
| in ARC and don't know, obviously*
| rfoo wrote:
| > You're saying the publicly available problem set isn't
| indicative of the distribution of the test set?
|
| Yes. From https://arcprize.org/guide:
| Please note that the public training set consists of
| simpler tasks whereas the public evaluation set is
| roughly the same level of difficulty as the private test
| set. The public training set is significantly
| easier than the others (public evaluation and private
| evaluation set) since it contains many "curriculum" type
| tasks intended to demonstrate Core Knowledge systems.
| It's like a tutorial level.
| advael wrote:
| Well, in this paragraph they seem to explain that their
| public evaluation set is meant to be indicative of the
| kind of jump in difficulty you can expect from the
| private test set. This to me implies that my guess is
| close: They're looking for models that can learn simple
| concepts and apply them to complex problems. Keeping the
| test set private seems to be an attempt at making it
| difficult to "cheat" at this by simply memorizing
| superficial details of the more complex problem set,
| which makes sense given that the whole point of this
| seems to be testing for systems that can use learned
| abstractions to tackle novel, out-of-distribution
| problems
|
| Like with our toy "algebra" examples, sure there's a lot
| of emphasis on repetition and rote in primary education
| on these subjects, and that's one way to get people more
| consistent at getting the calculations right, but to be
| frank I don't think it's the best way, or as crucial as
| it's made out to be. What someone really needs to
| understand about algebra is how the notation works and
| what the symbols mean. Like I can't unsee the concept of
| "+" as a function that takes two operands and starts
| counting for as many steps as one would in the right
| operand, starting at the value of the left operand. When
| looking at algebra, the process I go through relies on a
| bunch of conceptual frameworks, like "Anything in the set
| of all arabic numerals can be considered a literal
| value". "Anything in the roman alphabet is likely a
| variable". "Any symbol is likely an infix operator, that
| is, a function whose operands are on either side of it".
| Some of the concepts I'm using are just notational
| convention. At some point I memorized the set of arabic
| numerals, what they look like, what each of them means,
| how they're generally written in relation to each other
| to express quantities combinatorically. Some of the
| concepts are logical relations about quantities, or
| definitions of functions. But crucially, the form of
| these distillations makes them composable. If I didn't
| really understand what "+" does, then maybe someone could
| give me some really bad homework that goes
|
| 1 + 30 = 31
|
| 20 + 7 = 27
|
| 3 + 10 = 13
|
| And then present me the problem
|
| 20 + 10 + 3 = ?
|
| And I'd think the answer is
|
| 20 + 10 + 3 = 213
|
| That demonstrates _some model_ of how to do these
| calculations, but it doesn 't really capture all the
| important relationships the symbols represent
|
| We can have any number of objections to this training
| set. Like I wasn't presented with any examples of adding
| two-digit numbers together! OR even any examples where I
| needed to combine numbers in the same rank!
|
| Definitely all true. Probably mistakes we could make in
| educating a kid on algebraic notation too. It's really
| hard to do these things in a way that's both
| accomplishing the goal _and_ testable, quantifiable. But
| many humans demonstrate the ability to distill conceptual
| understanding of concepts without exhaustive examples of
| their properties, so that 's one of the things ARC seems
| to want to test. It's hard to get this perfectly right,
| but it's a reasonable thing to want
| blobbers wrote:
| The problem is there is no way to infer the right answer
| to 0! given the training. You need more context to learn
| it. Humans need more context. If you put that at the end
| of every grade 1 math test no student would get it right
| unless they had some context.
|
| Do grade 1 kids have AGI? (Haha)
|
| But seriously, all professions need to train in context
| to solve complex problems. You can train in adjacent
| realms and reason about problems but to truly perform,
| you need more training.
|
| A general surgeon might be better than an electrician as
| a vet, but that I'd rather have a veterinary surgeon
| operate on my dog.
|
| So some things are "AGI" able and other things need
| specific training.
| advael wrote:
| I think there's variance in people's degree of
| compositionality, as well as how quickly they can pick up
| on novel relationships. Testing "intelligence" in humans
| has always been kind of fraught in the first place, but
| any capability we may care to measure is going to permit
| degrees, and there will be some variance in humans on it.
| We should expect this. There's variance in goddam
| everything
|
| We should also expect machine learning systems to have
| somewhat different properties from human minds. Like
| computers are more likely to accomplish perfect recall,
| and we can scale the size of their memory and their
| processing speed. All these confounding variables can
| make it hard to make binary tests of a _capability_ ,
| which is really what ARC seems like it's trying to do.
| One such capability that AI researchers will often talk
| about is conceptual compositionality. People care about
| compositionality because it's a good way to demonstrate
| that an abstract model is being used to reason about a
| situation, which can be used in unseen but perhaps
| conceptually similar situations. This "generalization" or
| "abstraction" capability is really the goal, but it's
| hard to reason about how to test it, and "composition"
| (That is, taking a situation that's novel, but a
| straightforward application of two or more different
| abstractions the agent should already "know") is one more
| testable way to try to tease it out.
|
| As you point out, humans often fail this kind of test,
| and we can rightly claim that in those cases, they didn't
| correctly grasp the insight we were hoping they had.
| Testing distilled abstractions versus memorization or
| superficial pattern recognition isn't just important to
| AI research, it's also a key problem in lots of places in
| human education
| astromaniak wrote:
| > First, LLMs are not AGI.
|
| It's the most generic thing we have right now, right?
|
| > Never will be.
|
| If there is no other breakthrough anytime soon we can
| engineer AGI-like things around LLMs. I mean LLM trained
| to use different attachments. Which can be other models
| and algorithms. Examples will be image recognition models
| and databases for algorithms. Even now ChatGPT can use
| Bing search and Python interpreter. First steps done,
| others will follow. The result will be not a true AGI,
| but still a very capable system. And there is another
| factor. Next models can be trained on high quality data
| generated by current models. Instead of internet random
| garbage. This should improve their spacial and logical
| abilities.
| ec109685 wrote:
| There are similarities to the approach in this paper (though
| they trained a model from scratch):
| https://arxiv.org/pdf/2309.07062
|
| How well would an LLM trained with a huge number of examples do
| on this test? Essentially with enough attention, Goodhart's law
| will take over.
| sriku wrote:
| Part of the challenge I understood to be learning priors from
| the training set that can then be applied to an extended
| private test set. This approach doesn't seem to do any such
| "learning" on the go. So, supposing it accomplished 85% on the
| private test set, would it be construed to have won the prize
| with "we have AGI" being trumpeted out?
| jd115 wrote:
| Reminds me a bit of Genetic Programming as proposed by John
| Holland, John Koza, etc. Ever since GPT came out, I've been
| thinking of ways to combine that original idea with LLMs in
| some way that would accelerate the process with a more
| "intelligent" selection.
| lachlan_gray wrote:
| I'd love to hear more about this!
| lelanthran wrote:
| Maybe I am missing something, but to me this looks like "Let's
| brute-force on the training data".
|
| I mean, generating tens of thousands of possible solutions, to
| find one that works does not, to me, signify AGI.
|
| After all, the human solving these problem doesn't make 10k
| attempts before getting a solution, do they?
|
| The approach here, due to brute force, can't really scale: if a
| random solution to a _very simple_ problem has a 1 /10k chance
| of being right, you can't scale this up to non-trivial problems
| without exponentially increasing the computational power used.
| Hence, I feel this is brute-force.
| killerstorm wrote:
| 10000 samples are nothing compared to 2^100 possible outputs.
| It is absolutely, definitely not a "brute search". Testing a
| small fraction of possibilities (e.g. 0.000001%) is called
| heuristics, and that's what people use too.
|
| Please learn a bit of combinatorics.
|
| > After all, the human solving these problem doesn't make 10k
| attempts before getting a solution, do they?
|
| No. People have much better "early rejection", also human
| brain has massive parallel compute capacity.
|
| It's ridiculous to demand GPT-4 performs as good as a human.
| Obviously its vision is much worse and it doesn't have
| 'video' and physics priors people have, so it has to guess
| more times.
| lelanthran wrote:
| > 10000 samples are nothing compared to 2^100 possible
| outputs. It is absolutely, definitely not a "brute search".
| Testing a small fraction of possibilities (e.g. 0.000001%)
| is called heuristics, and that's what people use too.
|
| Brute searching literally means generating solutions until
| one works. Which is exactly what is being done here.
|
| > Please learn a bit of combinatorics.
|
| Don't be condescending - I understand the problem space
| just fine. Fine enough to realise that the problem was
| constructed specifically to ensure that "solutions" such as
| this just won't work.
|
| Which is why _this_ "solution" is straight-up broken
| (doesn't meet the target, exceeds the computationally
| bounds, etc).
|
| > It's ridiculous to demand GPT-4 performs as good as a
| human.
|
| Wasn't the whole point of this prize to spur interest in a
| new approach to learning? What does GPT-[1234] have to do
| with the contest rules? Especially since this solution
| broke those rules anyway?
|
| > Obviously its vision is much worse and it doesn't have
| 'video' and physics priors people have, so it has to guess
| more times.
|
| That's precisely my point - it has to _guess_. Humans aren
| 't guessing for those types of problems (not for the few
| that I saw anyway).
| ealexhudson wrote:
| I think to be clear, brute force generally means an
| iterative search of a solution space. I don't think
| that's what this system is doing, and it's not like it's
| following some search path and returning as early as
| possible.
|
| It's similar that a lot of wrong answers are being thrown
| up, but I think this is more like a probabilistic system
| which is being pruned than a walk of the solution space.
| It's much smarter, but not as smart as we would like.
| lelanthran wrote:
| > I think to be clear, brute force generally means an
| iterative search of a solution space.
|
| Sure, but not an exhaustive one - you stop when you get a
| solution[1]. Brute force does not require an exhaustive
| search in order to be called brute-force.
|
| GP was using the argument that because it is not
| exhaustive, it cannot be brute-force. That's the wrong
| argument. Brute-force doesn't have to be exhaustive to be
| brute-force.
|
| [1] Or a good enough solution.
| naasking wrote:
| A brute force search can be expected to find a solution
| after a more thorough search of the space of
| possibilities. If it really is only searching 0.000001%
| of that space before finding solutions, then some
| structure of the problem is guiding the search and it's
| no longer brute force.
| killerstorm wrote:
| I was a member of national ACM ICPC team, I studied
| algorithms for years. Brute force, heuristics, etc.
|
| You're 100% WRONG on everything you wrote. You don't get
| change established terminology just to denigrate the
| approach you don't like.
|
| It seems you are just biased against LLMs. But this test
| is as much a test for limits of LLM abilities as it is a
| quest for a new approach. It's open ended.
|
| And it's not like the author of the article claims he has
| AGI - he just shows the limits of LLM-based solutions.
| lelanthran wrote:
| > I studied algorithms for years.
|
| Who hasn't?
|
| > You're 100% WRONG on everything you wrote.
|
| Maybe you should update the wikipedia page, then all the
| other textbooks, that uses a definition of brute-force
| that matches my understanding of it.
|
| From https://en.wikipedia.org/wiki/Brute-force_search
|
| > Therefore, brute-force search is typically used when
| the problem size is limited, or when there are problem-
| specific heuristics that can be used to reduce the set of
| candidate solutions to a manageable size.
|
| Further, in the same page
| https://en.wikipedia.org/wiki/Brute-
| force_search#Speeding_up...
|
| > One way to speed up a brute-force algorithm is to
| reduce the search space, that is, the set of candidate
| solutions, by using heuristics specific to the problem
| class.
|
| I mean, the approach under discussion is literally
| _exactly_ this.
|
| Now, _Mr "ACM ICPC, studied algorithms for years"_,
| where's _your_ reference that reducing the solution space
| using heuristics results in a non-brute-force algorithm?
| baobabKoodaa wrote:
| This was extremely cringe worthy to read. You are
| confidently wrong about this. You are trying to redefine
| well established terminology. I don't care about whatever
| random wiki page you might find to "support your claims".
| Anybody who has worked a lot on algorithms (including
| myself) knows what brute force means and this is not it.
|
| Also: lol at your "who hasn't" comment. Because you
| clearly haven't.
| lelanthran wrote:
| > You are trying to redefine well established
| terminology.
|
| Reference? Link, even?
|
| > don't care about whatever random wiki page you might
| find to "support your claims".
|
| That isn't some "random wiki" page; that's the wikipedia
| page for this specific term.
|
| I'm not claiming to have defined this term, I'm literally
| saying I only agree with the sources for this term.
|
| > Also: lol at your "who hasn't" comment. Because you
| clearly haven't.
|
| Talk about cringe-worthy.
| baobabKoodaa wrote:
| > Reference? Link, even?
|
| Sure, here's definition for "brute force" from university
| textbook material written by pllk, who has taught
| algorithms for 20 years and holds a 2400 rating on
| Codeforces:
|
| https://tira.mooc.fi/kevat-2024/osa9/
|
| _" Yleispateva tapa ratkaista hakuongelmia on toteuttaa
| raakaan voimaan (brute force) perustuva haku, joka kay
| lapi kaikki ratkaisut yksi kerrallaan."_
|
| edit:
|
| Here's an English language book written by the same
| author, though the English source does not precisely
| define the term:
|
| https://cses.fi/book/book.pdf
|
| In chapter 5:
|
| _" Complete search is a general method that can be used
| to solve almost any algorithm problem. The idea is to
| generate all possible solutions to the problem using
| brute force ..."_
|
| And a bit further down chapter 5:
|
| _" We can often optimize backtracking by pruning the
| search tree. The idea is to add "intelligence" to the
| algorithm so that it will notice as soon as possible if a
| partial solution cannot be extended to a complete
| solution. Such optimizations can have a tremendous effect
| on the efficiency of the search."_
|
| Your mistake is that you for some reason believe that
| _any_ search over solution space is a brute force
| solution. But there are many ways to search over a
| solution space. A "dumb search" over solution space is
| generally considered to be brute force, whereas a "smart
| search" is generally not considered to be brute force.
|
| Here's the Codeforces profile of the author:
| https://codeforces.com/profile/pllk
|
| edit 2:
|
| Ok now I think I understand what causes your confusion.
| When an author writes "One way to speed up a brute-force
| algorithm ..." you think that the algorithm can still be
| called "brute force" after whatever optimizations were
| applied. No. That's not what that text means. This is
| like saying "One way to make a gray car more colorful is
| by painting it red". Is it still a gray car after it has
| been painted red? No it is not.
| killerstorm wrote:
| You're asking for a definition of exhaustive search.
| Exhaustive search, by definition, goes through the entire
| search space. That's what word exhaustive means.
|
| For a reference, check Cormen's "Introduction to
| Algorithms". Every mention of brute-force search is
| specifically to exhaustive search is which not feasible
| for bigger spaces.
|
| > I mean, the approach under discussion is literally
| exactly this.
|
| It's literally not. It DOES NOT REDUCE the candidate set.
| It generates most likely candidates, but it doesn't
| reduce anything.
|
| You lack basic understanding. Solutions are pixels grids,
| not Python programs. There's no search over pixel grids
| in the article. Not every search is exhaustive search.
|
| This is like saying theoretical physicists are "brute-
| forcing" physics by generating candidate theories and
| testing them. Ridiculous.
| YeGoblynQueenne wrote:
| Ah, give it a rest. That's not "frontier AI research", neither
| is it any kind of reasoning. It's the dumbest of the dumb
| possible generate-and-test approach that spams a fire hose of
| Python programs until it hits one that works. And still it gets
| only 50% on the public eval.
|
| How many thousands of Python programs does a human need to
| solve a single ARC task? That's what you get with reasoning:
| you don't need oodles of compute and boodles of sampling.
|
| And I'm sorry to be so mean, but ARC is a farce. It's supposed
| to be a test for AGI but its only defense from a big data
| approach (what Francois calls "memorisation") is that there are
| few examples provided. That doesn't make the tasks hard to
| solve with memorisation it just makes it hard for a human
| researcher to find enough examples to solve with memorisation.
| Like almost every other AI-IQ test before it, ARC is testing
| for the wrong thing, with the wrong assumptions. See the
| Winograd Schema Challenge (but not yet the Bongard problems).
| jononor wrote:
| Do you have any suggestions for a better approach of testing
| artificial intelligence? I mean, in a way that allows
| comparing different approaches and being a reasonable metric
| of progress.
| YeGoblynQueenne wrote:
| I don't. I'm guessing -and it's nothing but a guess- that
| for every problem that can be solved with intelligence
| there exists a solution that does not require intelligence.
| I'm guessing in other words that intelligence is the
| ability to come up with solutions to arbitrary problems. If
| that's true then there's no way to test for intelligence by
| looking at the performance of a system at any particular
| task, or any finite set of tasks, and so there's no way to
| create a "test for intelligence".
|
| My guess is supported by the experience that, in AI
| research, every time someone came up with a plausible test
| for intelligence, an AI system eventually passed the test
| only to make it clear that the test was not really testing
| intelligence after all (edit: I don't just mean formal
| tests; e.g. see how chess used to "require intelligence"
| right up until Deep Blue vs Kasparov).
|
| Some people see that as "moving the goalposts" and it's
| certainly frustrating but the point is that we don't know
| what intelligence is, exactly, so it's very hard to test
| for its existence or not, or to measure it.
|
| My preference would be for everyone in AI research to
| either stop what they're doing and try to understand what
| the hell intelligence is in the first place, to create a
| theory of intelligence so that AI can be a scientific
| subject again, or to at least admit they're not interested
| in creating artificial intelligence. I, for example, am
| not, but all my background is in subjects that are
| traditionally labelled "AI" so I have to suck it up, I
| guess.
| Nimitz14 wrote:
| You're basically paraphrasing fchollet's paper on
| intelligence and what he talked about in his most recent
| podcast appearance with dwarkesh.
| machiaweliczny wrote:
| Do you accept such solutions as legit? It's obviously is easier
| to generate program that to make prompt that will solve it
| data_maan wrote:
| It's not _that_ novel. Others have implemented this approach ,
| in the context of mathematics.
|
| Already the 2021 paper Drori (and many papers since) did
| similar things.
|
| It's a common idea in this space...
| asperous wrote:
| Having tons of people employ human ingenuity to manipulate
| existing LLMs into passing this one benchmark kind of defeats the
| purpose of testing for "AGI". The author points this out as it's
| more of a pattern matching test.
|
| Though on the other hand figuring out which manipulations are
| effective does teach us something. And I think most problems boil
| down to pattern matching, creating a true, easily testable AGI
| test may be tough.
| janalsncm wrote:
| Perhaps if we don't know how to create an evaluation that can't
| be "gamed" it tells us something about how special our
| intelligence really is?
| lucianbr wrote:
| I don't know how to create a liver, or test one, so what does
| that say about my liver? Pretty much nothing.
| opdahl wrote:
| Wouldn't the real AGI test be that an AI would be able to do
| what the author did here and write this blog post?
| atroche wrote:
| Yep, but a float is more useful than a bool for tracking
| progress, especially if you want to answer questions like
| "how soon can we expect (drivers/customer support
| staff/programmers) to lose their jobs?"
|
| Hard to find the right float but worth trying I think.
| opdahl wrote:
| I agree, but it does seem a bit strange that you are
| allowed to "custom-fit" an AI program to solve a specific
| benchmark. Shouldn't there be some sort of rule that for
| something to be AGI it should work as "off-the-shelf" as
| possible?
| soist wrote:
| If OpenAI had an embedded python interpreter or for that
| matter an interpreter for lambda calculus or some other
| equally universal Turing machine then this approach would
| work but there are no LLMs with embedded symbolic
| interpreters. LLMs currently are essentially probability
| distributions based on a training corpus and do not have
| any symbolic reasoning capabilities. There is no
| backtracking, for example, like in Prolog.
| killerstorm wrote:
| I won't be surprised if GPT-5 would be able to do it: it
| knows that it's LLM, so it knows its limitations. It can
| write code to pre-process input in a format which is better
| understood, etc.
|
| https://chatgpt.com/share/2fde1db5-00cf-404d-9ae5-192aa5ac90.
| ..
|
| GPT-4 created a plan very similar to the article, i.e. it
| also suggested using Python to pre-process data. It also
| suggested using program synthesis. So I'd say it's already
| 90% there.
|
| > "Execute the synthesized program on the test inputs."
|
| > "Verify the outputs against the expected results. If the
| results are incorrect, iteratively refine the hypotheses and
| rules."
|
| So people saying that it's ad-hoc are wrong. LLMs know how to
| solve these tasks, they are just not very good at coding, and
| iterative refinement tooling is in infancy.
| sheeshkebab wrote:
| Show me a test and I'll show you a neural network that passes
| it... used to be an saying.
| worstspotgain wrote:
| Let me play devil's advocate for a second. Let's suppose that
| with LLMs, we've actually invented an AGI machine that also
| happens to produce useful textual responses to a prompt.
|
| This would sound more far-fetched if we knew exactly how they
| work, bit-by-bit. We've been training them statistically, via
| the data-for-code tradeoff. The question is not yet
| satisfactorily answered.
|
| In this hypothetical, for every accusation that an LLM passes a
| test because it's been coached to do so, there's a counter that
| it was designed for "excessively human" AGI to begin with,
| maybe even that it was designed for the unconscious purpose of
| having humans pass it preferentially. The attorney for the
| hypothetical AGI in the LLM would argue that there are tons of
| "LLM AGI" problems it can solve that a human would struggle
| with.
|
| Fundamentally, the tests are only useful insofar as they let us
| improve AI. The evaluation of novel approaches to pass them
| like this one should err in the approaches' favor, IMO. A
| 'gotcha' test is the least-useful kind.
| vlovich123 wrote:
| There's every reason to believe that AGI is meaningfully
| different from LLMs because humans do not take anywhere near
| this amount of training data to create inferences (that and
| executive planning and creative problem solving are clear
| weak spots in LLMs)
| og_kalu wrote:
| >There's every reason to believe that AGI is meaningfully
| different from LLMs because humans do not take anywhere
| near this amount of training data to create inferences
|
| The human brain is millions of years of brute force
| evolution in the making. Comparing it to a transformer or
| any other ANN really which essentially start from scratch
| relatively speaking doesn't mean much.
| infgeoax wrote:
| Plus it's unclear if the amount of data used to "train" a
| human brain is really less than what GPT4 used. Imagine
| all the inputs from all the senses of a human over a
| lifetime: the sound, light, touches, interactions with
| peers, etc.
| Jensson wrote:
| But that is of little help when you want to train an LLM
| to do the job at your company. A human requires just a
| little bit of tutorials and help, an LLM still require an
| unknown amount of data to get up to speed since we
| haven't reached that level yet.
| infgeoax wrote:
| Yeah humans can generalize much faster than LLM with far
| fewer "examples" running on sandwiches and coffee.
| logicchains wrote:
| >Yeah humans can generalize much faster than LLM with far
| fewer "examples" running on sandwiches and coffee.
|
| This isn't really true. If you give an LLM a large prompt
| detailing a new spoken language, programming language or
| logical framework with a couple examples, and ask it to
| do something with it, it'll probably do a lot better at
| it than if you just let an average human read the same
| prompt and do the same task.
| alchemist1e9 wrote:
| Don't forget all the lifetimes of all ancestors as well.
| A lot of our intelligence is something we are born with
| and a result of many millions of years of evolution.
| visarga wrote:
| How many attempts have there been for humans to solve math
| or science outstanding problems? We're also kind of
| spamming with ideas until one works out
| vlovich123 wrote:
| I'll give you as much time as you want with an LLM and am
| 100% sure that it won't solve a single outstanding
| complex math problem.
| danielbln wrote:
| I can say the same about myself, and I would probably
| consider myself generally intelligent.
| vlovich123 wrote:
| There's a meaningful difference between a silicon
| intelligence and an organic one. Every silicon
| intelligence is closer to an equally smart clone whereas
| organic ones have much more variance (not to mention
| different training).
|
| Anyway, my point was that humans butter direct their
| energy than randomly spamming ideas, at least with the
| innovation of the scientific method. But an LLM struggles
| deeply to perform reasoning.
| coolspot wrote:
| > I'll give you as much time as you want with an LLM
|
| With infinite amount of time you can LLM brute force
| whole search space. Infinite monkeys with typewriters.
| bongodongobob wrote:
| Our compute architecture has been brute forced via an
| revolutionary algorithm over a billion years. An LLM
| approaching our capabilities in like a year is pretty
| fucking good.
| yieldcrv wrote:
| its LLM grade school. let them cook, train these things to
| match utility in our world. I'm not married to the "AGI" goal
| if there is other utility along the way.
| trott wrote:
| Francois Chollet says LLMs do not learn in-context. But Geoff
| Hinton says LLMs' few-shot learning compares quite favorably with
| people!
|
| https://www.youtube.com/watch?v=QWWgr2rN45o&t=46m20s
|
| The truth is in the middle, I think. They learn in-context, but
| not as well as humans.
|
| The approach in the article hides the unreliability of current
| LLMs by generating thousands of programs, and still the results
| aren't human-level. (This is impressive work though -- I'm not
| criticizing it.)
| hackpert wrote:
| I'm not sure how to quantify how quickly or well humans learn
| in-context (if you know of any work on this I'd love to read
| it!)
|
| In general, there is too much fluff and confusion floating
| around about what these models are and are not capable of
| (regardless of the training mechanism.) I think more people
| need to read Song Mei's lovely slides[1] and related work by
| others. These slides are the best exposition I've found of neat
| ideas around ICL that researchers have been aware of for a
| while.
|
| [1]
| https://www.stat.berkeley.edu/~songmei/Presentation/Algorith...
| TheDudeMan wrote:
| "Vision is an especially large weakness."
|
| But you can have GPT write code to reliably convert the image
| grid into a textual representation, right? And code to convert
| back to image and auto-verify.
| p1esk wrote:
| GPT-4o might not have been trained on a sufficiently large
| amount of visual data to develop advanced spatial intelligence.
| Perhaps it needs to see a lot more images, or perhaps it needs
| to be trained differently (e.g. predict the next frame in a
| video). I suspect SORA has more spatial intelligence internally
| than 4o.
| gibsonf1 wrote:
| Isn't 50% kind of a failing grade?
| p1esk wrote:
| It's only been a week since the million dollar prize was
| offered. Let's see what the SOTA is in a month.
| badrunaway wrote:
| When we talk about system 2; is it possible that [generating
| large number of programs; evaluating them of the task; choosing
| top K outcomes; feeding it back to Neural net] can act as system
| 2 for a AGI? Isn't that how we think intelligently as well- by
| making lot of hypothesis internally and evaluating them - and
| updating our model?
| awwaiid wrote:
| I think it's more like humans are a chaotic choir of subsystems
| all doing their thing and tossing up their directives until
| some sort of "win" happens or the volume is loud enough in some
| direction that it then gets reverse engineered into a
| "thought". But yes.
| badrunaway wrote:
| like darwin selection between the subsystem approaches? Put
| in a lot of different kind of LLMs and let them play the same
| game inside with each other.. whosoever wins the simulation
| is allowed to externally present the approach... something
| like that?
| spencerchubb wrote:
| Possibly
|
| I think we need those pieces, and also a piece for determining
| hypotheses in an efficient manner. Monte Carlo Tree Search
| could be that piece. Probabilistically choose a node to search,
| and then backpropagate the probabilities back to the root node.
| badrunaway wrote:
| Intuitively I feel efficiency is the outcome of existing
| world model.. approach can look like yours - I don't see why
| there has not been efforts on scaling monte carlo tree search
| for extending the existing world model via tree search. My
| guess is that it would diverge to hallucinations too fast
| because it doesn't have a strong logical building block
| already
| bshanks wrote:
| Yes, I think it's very possible that human brains unconsciously
| generate-and-test surprisingly large numbers of small candidate
| programs when solving a problem.
|
| This approach is
| https://en.wikipedia.org/wiki/Embarrassingly_parallel, which is
| a good fit for biological neural architectures, which have very
| many computing nodes but each node is very slow (compared to
| electronic computer CPUs/GPUs).
| Imnimo wrote:
| To me the big take-aways here are:
|
| 1) Most of the heavy lifting is being done by search. We're
| talking about having the LLM generate _thousands_ of candidate
| solutions, and they 're mostly bad enough that "just pick the
| ones that get kinda close on the examples" is a meaningful
| operation.
|
| 2) More samples improves performance despite the fact that
| GPT-4o's vision is not capable of parsing the inputs. I'm curious
| how much performance would degrade if you shuffled the images
| passed to the model (but used the correct images when evaluating
| which candidates to keep).
|
| 3) It's definitely true that the LLM has to be giving you
| something more than random programs. At the very least, the LLM
| knows how to craft parsimonious programs that are more likely to
| be the solution. It may be that it's providing more than that,
| but it's not clear to me exactly how much information on the
| correct search space is coming from the hand-crafted examples in
| the prompt.
|
| Overall, the work to get this far is very impressive, but it
| doesn't really move the needle for me on whether GPT-4 can do ARC
| puzzles. It does, however, show me that search is surprisingly
| powerful on this task.
| bearjaws wrote:
| Seems that Arc-AGI is more flawed rather than GPT-4o is more AGI.
|
| Maybe a AI version of Hanlons Razor. Never attribute to AGI what
| could be easily explained by being in the training set.
| bjornsing wrote:
| Can we be sure GPT-4o hasn't been trained on the public test set?
| ivalm wrote:
| It has been trained on public test set as it's on github
| free_bip wrote:
| This isn't necessarily true, OpenAI isn't open about how
| their data cleaning process works.
| atleastoptimal wrote:
| I'll say what a lot of people seem to be denying. GPT-4 is an
| AGI, just a very bad one. Even GPT-1 was an AGI. There isn't a
| hard boundary between non AGI and AGI. A lot of people wish there
| was so they imagine absolutes regarding LLM's like "they cannot
| create anything new" or something like that. Just think: we
| consider humans a general intelligence, but obviously wouldn't
| consider an embryo or infant a general intelligence. So at what
| point does a human go from not generally intelligent to generally
| intelligent? And I don't mean an age or brain size, I mean suite
| of testable abilities.
|
| Intelligence is an ability that is naturally gradual and emerges
| over many domains. It is a collection of tools via which general
| abstractive principles can be applied, not a singular universally
| applicable ability to think in abstractions. GPT-4, compared to a
| human, is a very very small brain trained for the single purpose
| of textual thinking with some image capabilities. Claiming that
| ARC is the absolute market of general intelligence fails to
| account for the big picture of what intelligence is.
| blharr wrote:
| The "general" part of AGI implies it should be capable across
| all types of different tasks. I would definitely call it real
| Artificial Intelligence, but it's not general by any means.
| FeepingCreature wrote:
| It's capable of _attempting_ all types of different tasks.
| That is a novel capability on its own. We 're used to GPT's
| amusing failures at this point, so we forget that there is
| absolutely no input you could hand to a chess program that
| would get it to try and play checkers.
|
| Not so with GPT. It will try, and fail, but that it tries at
| all was unimaginable five years ago.
| dahart wrote:
| Its amusing to me how the very language used to describe
| GPT anthropomorphizes it. GPT wont "attempt" or "try"
| anything on its own without a human telling it what to try,
| it has no agenda, no will, no agency, no self-reflection,
| no initiative, no fear, and no desire. It's all A and no I.
| FeepingCreature wrote:
| Do you agree that "there is absolutely no input you could
| hand to a chess program that would get it to try and play
| checkers", but there _is_ an input you can hand to GPT-3+
| that will get it to try and play pretty much any game
| imaginable, so long as you agree that its attempt will be
| very poor?
|
| I don't want to get into the weeds on what intelligence
| is or what "attempt" means or "try" means (you can
| probably guess I disagree with your position), but do you
| have a disagreement on pure input/output behavior? Do you
| disagree that if I put adequate words in, words will come
| out that will resemble an attempt to do the task, for
| nearly any task that exists?
| dahart wrote:
| You're trying to avoid addressing my point. What can GPT
| do that's interesting without a human in the loop doing
| the prompting?
|
| Lol "very poor". You're attempting to argue that if
| there's any output at all in response to an input prompt,
| then GPT is "trying" and showing signs of intelligence,
| no matter what the output is. By this logic, you
| contradicted yourself: the chess engine can play
| checkers, poorly. By this logic, asking the sky to play a
| game means the sky is trying because it changes, or
| asking a random number generator to play a game means it
| resembles an attempt to play because there is "very poor"
| output.
|
| There are lots of games GPT can't play, like hide-and-
| seek, tag, and tennis. Playing a game means playing by
| the rules of the game, giving coherent output, and trying
| to win. GPT can't play games it hasn't seen before, and
| no I don't agree that "very poor" output counts. It
| doesn't (currently) learn the rules from your prompts;
| you can't teach it to play a new game by talking to it,
| and the "very poor" output from a game it wasn't trained
| on will never improve. And, to my actual point, GPT will
| not play any games at all unless you ask it to.
| CamperBob2 wrote:
| _What can GPT do that's interesting without a human in
| the loop doing the prompting?_
|
| Understand what the human in the loop doing the prompting
| is asking for, for one thing.
|
| The magical aspects of LLMs are on the _input_ side, not
| the output.
| dahart wrote:
| This probably isn't what you meant, but if the magic is
| on the input side, then everything interesting about
| interacting with GPT is being provided by the human and
| not GPT.
|
| We don't have any strong evidence that GPT "understands"
| its input in general. We absolutely have examples of GPT
| failing to understand some inputs (and not knowing it,
| and insisting on bogus output). And we know for a fact
| that it was designed and built to produce statistically
| plausible output. GPT is a mechanical device designed by
| humans to pass the Turing test. We've designed and built
| something that is exceptionally good at making humans
| believe it is smarter than it is.
| CamperBob2 wrote:
| _We've designed and built something that is exceptionally
| good at making humans believe it is smarter than it is._
|
| Yep, and even ELIZA could do that, to some extent. But at
| some point you'll need to define what "understanding"
| means, and explain why an LLM isn't doing it.
| dahart wrote:
| Totally, and that's a fair point. I don't know what
| understanding means, not enough to prove an LLM can't,
| anyway, and I think nobody has a good enough definition
| yet to satisfy this crowd. But I think we can make
| progress with nothing more than the dictionary definition
| of "understand", which is the ability to perceive and
| interpret. I think we can probably agree that a rock
| doesn't understand. And we can probably also agree that a
| random number generator doesn't understand. The problem
| with @FeepingCreature's argument is that the quality of
| the response does matter. The ability for a machine
| that's specifically designed to wait for input and then
| provide an output, to then provide a low quality
| response, doesn't demonstrate any more intelligence than
| a bicycle... right? I don't know where the line is
| between my random writer Markov chain text generator from
| college and today's LLMs. I'm told transformers are
| fundamentally the same and just have an adaptive window
| size. More training data then is the primary difference.
| So then we are saying Excel's least-squares function
| fitter does not understand, unless the function has a
| billion data points? Or, if there's a line, what does it
| look like and where is it?
| lupusreal wrote:
| That's not an interesting argument, all you're doing is
| staking out words that are reserved for beings with souls
| or something. It's like saying submarines can't swim.
| It's an argument about linguistics, not capabilities.
| dahart wrote:
| Disagree. The ability to "try" is a capability, and GPT
| doesn't have it. Everything in the world that we've
| called "intelligent" up to this point has had autonomy
| and self motivation, and GPT doesn't have those things.
| GPT doesn't grow and doesn't learn from its mistakes. GPT
| won't act without a prompt, this is a basic fact of its
| design, and I'm not sure why people are suddenly confused
| about this.
| theptip wrote:
| This seems to be so broad a definition as to no longer mean
| anything useful.
|
| People in general are interested in capabilities or economic
| impact, and GPT-2 cleared no notable thresholds in those
| regards.
|
| I prefer the exact opposite approach: let's use a strict
| definition, and have levels to make it really explicit what we
| are talking about.
|
| Here is a good one:
|
| "Levels of AGI for Operationalizing Progress on the Path to
| AGI"
|
| https://arxiv.org/abs/2311.02462
| sigmoid10 wrote:
| People will never agree on this. We've known about the
| concept of intelligence for much, much longer than computers
| have been around and we still don't have a common definition
| or set of rules to check. That also makes things like
| consciousness and death pretty hard to define in medicine,
| leading to inconsistent rules across jurisdictions. For AGI
| in particular, I guarantee you that no matter which test gets
| beaten, the majority of humanity will always just move the
| goalpost and claim it's not "real" AGI because "reasons."
| Because the opposite would mean they have to admit that they
| are now the lesser intelligence on the planet.
| surfingdino wrote:
| > GPT-4 is an AGI, just a very bad one.
|
| Then stop selling it as a tool to replace humans. A fast moving
| car breaking through a barrier and flying off the cliff could
| be called "an airborne means of transportation, just a very bad
| one" yet nobody is suggesting it should replace school busses
| if only we could add longer wings to it. What the LLM community
| refuses to see is that there is a limit to the patience and the
| financing the rest of the world will grant you before you're
| told, "it doesn't work mate."
|
| > So at what point does a human go from not generally
| intelligent to generally intelligent?
|
| Developmental psychology would be a good place to start looking
| for answers to this question. Also, forgetting scientific
| approach and going with common sense, we do not allow young
| humans to operate complex machinery, decide who is allowed to
| become a doctor, or go to jail. Intelligence is something that
| is not equally distributed across the human population and some
| of us never have much of it, yet we function and have a role in
| society. Our behaviour, choices, preferences, opinions are not
| just based on our intelligence, but often on our past
| experiences and circumstances. It is also not the sole quality
| we use to compare ourselves against each other. A not very
| intelligent person is capable of making the right choices (an
| otherwise obedient soldier refusing to press the button and
| blow up a building full of children); similarly, a highly
| intelligent person can become a hard-to-find serial criminal (a
| gynecologist impregnating his patients).
|
| What intelligent and creative people hold against LLMs is not
| that they replace them, but that they replace them with a shit
| version of them relegating thousands of years of human progress
| and creativity to the dustbin of the models and layers of
| tweaks to the output that still produce unreliable crap. I
| think the person who wrote this sign summed it up best
| https://x.com/gvanrossum/status/1802378022361911711
| bongodongobob wrote:
| In response to the sign: then learn to code or make art that
| is better than AI art.
|
| It's an existential complaint. "Why won't the nerds make
| something for meeeee." Do it yourself. Make that robot.
|
| Sucks to think that you're not that special. Most art isn't.
| Most music isn't. Any honest artist will agree. Most
| professional artists are graphic designers, not brilliant
| once in a generation visionaries. It's the new excuse for
| starving artists. AI or no, they'd still be unsuccessful.
| That's the way it's always been.
| earthnail wrote:
| While that is 100% true, the real problem is that the risk
| of finding out whether you can make special art has
| significantly increased. Previously if you didn't make it
| as an artist, you could still earn money with other art
| related tasks - in graphics, many became illustrators. In
| music, people made music for ads.
|
| That plan B is now going away, and a music career will be
| much more like a sports career: either you make it in
| football, or you need to find another career where your
| football skills won't be very useful.
|
| That is obviously scary for many.
| surfingdino wrote:
| Artists who make it usually have a legend, a story to
| tell or be told by their friends, associates, agents,
| publishers, gallerists, etc. That story has a human
| dimension that touches the rest of us and we somehow
| connect to it. Van Gogh cut of his ear, we still keep
| talking about it and wondering why? There is nothing AI
| can tell us about itself, its "art". The artistic
| struggle with AI is not about expressing your vision on a
| canvas in a way that makes others feel what you want them
| to feel but about forcing it to generate something it is
| incapable of generating or programmed not to generate. We
| got to the point where we are given crayons programmed to
| not draw the things others do not want them to draw or to
| draw HR-approved version of what the artist wants to
| draw. The future is now and it's shit.
| latexr wrote:
| Using van Gogh as an example of "artists who make it" is
| insane.
|
| Which I guess is appropriate, because he was literally
| crazy. He suffered from psychotic episodes and delusions
| and died from suicide, depressed and in poverty.
|
| That's the opposite of "making it". It's zero consolation
| that people like his work now, he never even knew.
| surfingdino wrote:
| > Using van Gogh as an example of "artists who make it"
| is insane.
|
| So is building a tool that will only generate "approved"
| art. We need to be able to express our idea, feelings,
| our perception of the world in ways that do not fit
| corporate standards of text, audio, or visual
| communication. It's part of being human.
| latexr wrote:
| > So is building a tool that will only generate
| "approved" art.
|
| And so is eating ice cream with your forehead. Are we
| just doing non sequiturs now? I didn't defend image
| generation tools _in the slightest_.
|
| > We need to be able to express our idea, feelings, our
| perception of the world in ways that do not fit corporate
| standards of text, audio, or visual communication.
|
| I agree. My point started and ended with "van Gogh in an
| awful example when talking about artist who 'made it'".
| That's it. There is nothing in there to be extrapolated
| to AI or any other subject.
| bongodongobob wrote:
| So don't use the tool or make your own. You don't get to
| tell people what to use or like.
| intended wrote:
| The sign reads (paraphrased): "I want AI to do my dishes so
| I can do art. Not do art so I can do my dishes"
|
| Your response is "learn to art" "The nerds dont owe you
| anything." "Most of You would be unsuccessful anyway"
|
| You brought in absolutely unrelated items.
|
| 1) Learn art - that is baked into what the Sign is saying.
| There is no Terminal Point for being an artist.
|
| 2) Nerds dont... - Where nerds come in as a class for this
| conversation?
|
| 2.1) if you can speak for all nerds, please note that I
| sure as heck dont want Warhammer 40k, I want Star Trek.
|
| 3) Most would be unsuccessful - so what?
|
| Are they happy practicing their craft? Do they have the
| choice to spend their time on those pursuits and enrich
| their lives, and share their joys with others around them?
| atleastoptimal wrote:
| > What the LLM community refuses to see is that there is a
| limit to the patience and the financing the rest of the world
| will grant you before you're told, "it doesn't work mate."
|
| The point about LLM's is they may have a lot of drawbacks
| right now but they're improving at a rapid pace. They already
| are very useful. There are hundreds of stories coming out of
| companies effectively leveraging them to replace workers in
| many natural-language related tasks. They're far more useful
| than a car that goes off a cliff.
|
| Nobody more useful than an LLM is being effectively replaced
| by an LLM. Those few companies that jump the gun too early
| are suffering for it.
|
| >That sign
|
| We already have dishwashers and washing machines. Companies
| are working on making humanoid robots that can do those
| things, it's just that it's harder to develop a fully-fledged
| embodied humanoid than it is to create the diffusion models
| and LLM's being used today. It's not some conspiracy to let
| AI do all the fun stuff first.
|
| Nobody is preventing anyone from making art or writing
| poetry. If someone finds value in AI art or writing, either
| you have to accept that they weren't the audience member you
| wanted, or you have to accept that your ability to be
| creative is a learnable algorithm same as anything else.
| empath75 wrote:
| > Then stop selling it as a tool to replace humans
|
| I don't understand why people assume that the purpose of any
| tool is to "replace humans". Automation doesn't replace
| humans and never has and never will. It simply does certain
| tasks that humans used to do, freeing people up to do
| different tasks. There is not a limited amount of work that
| can be done, there isn't a limited amount of _creative_ work
| that can be done. Even if AIs were good enough to do every
| creative task done by humans today (and they aren't and won't
| be any time soon), that doesn't mean that humans will have
| nothing of value to produce, or that humans will have been
| "replaced". There is always going to be work for humans to
| do, even in a universe where AI have super human capabilities
| at all tasks.
|
| In particular, human beings strongly value the opinions and
| creative output of _human beings_ simply for the reason that
| they are human and similar to them. That will never change,
| no matter how intelligent that AIs get.
| surfingdino wrote:
| > I don't understand why people assume that the purpose of
| any tool is to "replace humans".
|
| Because of this
| https://news.ycombinator.com/item?id=40070566
| Tepix wrote:
| The definition of AGI that i am familiar with is that it can do
| all (digital) tasks a human can do _at the level of an average
| human_.
|
| As long as this level hasn't been achieved in all domains, it
| isn't AGI.
| lupusreal wrote:
| This seems like a problematic standard. For one, it's very
| fuzzy. A human who is top 51% and one who's top 49% are very
| similar to each other and could probably swap places
| depending on how they're feeling that day; there's nothing
| fundamentally different going on in their heads. Even at the
| further ends of the scale, humans have essentially the same
| kind of brains and minds as each other, some more capable
| than others but still all belonging to the same category of
| thinking things. Your AGI definition bifurcates the human
| population into those that possess general intelligence and
| those who don't, but this seems hard to justify. At least,
| hard to justify when drawn there. If you put the line at
| profound mental retardation where a person can no longer
| function in society, that would make more sense. A slightly
| below average human may not be exceptional in any regard but
| I think they still possess what must be regarded as _general_
| intelligence.
|
| Furthermore, you're counting cases where humans do things the
| computer cannot but ignoring cases where the computer does
| things humans cannot. For instance, I doubt any human alive,
| let alone _average humans_ can give reasonable explanations
| for short snippets of computer code in as many languages as
| GPT-4o, or formulate poetry in as many styles on arbitrary
| topics, or rattle off esoteric trivia and opinions about
| obscure historic topics, .... I think you get the point. It
| has already surpassed average human abilities in many
| categories of intellectually challenging tasks, but with your
| definition if it fails at even one task an average human can
| do, then it lacks "general intelligence."
|
| I suggest that your definition is one for "AHI" (Average
| Human Intelligence), not one for "AGI" (Artificial _General_
| Intelligence.)
| dr_dshiv wrote:
| Totally agree. So does Peter Norvig:
| https://www.noemamag.com/artificial-general-intelligence-is-...
| lucianbr wrote:
| "A car is a plane, just a very bad one. It can do the cruising
| down the runway part, only the flying part is missing".
| Fatalist_ma wrote:
| Someone could say that, but we know and we can prove that a
| car is not a plane. An example of a bad plane would be the
| Wright Flyer.
| killerstorm wrote:
| Yes. GPT-3 was a clear AGI signal: "language models are few-
| shot learners". I.e. they can figure a pattern from few
| examples to apply it to something useful. That's general
| intelligence.
|
| But people choose to be in denial.
| whiplash451 wrote:
| The article jumps to the conclusion that "Given that current LLMs
| can perform decently well on ARC-AGI" after having used multiple
| hand-crafted tricks to get to these results, including "I also
| did a small amount of iteration on a 100 problem subset of the
| public test set" which is hidden in the middle of the article and
| not mentioned in the bullet list at the top.
|
| Adding the close-to ad-hominem attack on Francois Chollet with
| the comics at the beginning (Francois never claimed to be a
| neuro-symbolic believer), this work does a significant disservice
| to the community.
| z7 wrote:
| >Francois never claimed to be a neuro-symbolic believer
|
| His response:
|
| "This has been the most promising branch of approaches so far
| -- leveraging a LLM to help with discrete program search, by
| using the LLM as a way to sample programs or branching
| decisions. This is exactly what neurosymbolic AI is, for the
| record..."
|
| "Deep learning-guided discrete search over program space is the
| approach I've been advocating, yes... there are many different
| flavors it could take though. This is one of them (perhaps the
| simplest one)."
|
| https://x.com/fchollet/status/1802773156341641480
| YeGoblynQueenne wrote:
| That kind of neuro-symbolic AI is a bit like British cuisine:
| place two different things next to each other in the same
| plate, like bangers and mash, and call it "a dish".
|
| Nope. This is neurosymbolic AI:
|
| _Abductive Knowledge Induction From Raw Data_
|
| https://www.doc.ic.ac.uk/~shm/Papers/abdmetarawIJCAI.pdf
|
| That's a symbolic learning engine trained in tandem with a
| neural net. The symbolic engine is learning to label examples
| for the neural net that learns to label examples for the
| symbolic engine. I call that cooking!
|
| (Full disclosure: the authors of the paper are my thesis
| advisor and a dear colleague).
| whiplash451 wrote:
| Indeed. Francois Chollet himself said during his interview
| with Dwarkesh that he is not against LLMs and in fact
| believes that the long-term solution mixes LLMs with
| something else which has not been discovered yet (his bet is
| on discrete program search but is open to anything else).
|
| Pitching him against LLMs in such a binary fashion is
| deceiving and unfair.
| bogtog wrote:
| For what it's worth, the comic is based on a well-known meme,
| and the author must've wanted to stick to the format:
| https://media.licdn.com/dms/image/D4E10AQFryt0thryEeA/image-...
| killerstorm wrote:
| I think this work is great.
|
| A lot of top researchers claim that obvious deficiencies in LLM
| training are fundamental flaws in transformer architecture, as
| they are interested in doing some new research.
|
| This work show that temporary issues are temporary. E.g. LLM is
| not trained on grid inputs, but can figure things out after
| preprocessing.
| whiplash451 wrote:
| My claim is _not_ that this work is not useful. But however
| "great" your work is, misleading on the steps you took during
| your experiments and overselling your results is never a
| valid approach in research.
| killerstorm wrote:
| This is a blog post, sir. All details are written down.
| He's very clear about methods, it seems you're 1) biased;
| 2) have too high standards for blog posts.
| kalkin wrote:
| The comic at the beginning paints the "stack more layers" LLM
| people as clowns, not neurosymbolic people or by proxy Chollet.
| Yes, it suggests the "stack more layers" approach works anyway,
| but in a self-deprecating way...
|
| If this article wanted to attack Chollet, it could have made
| more hay out of another thing that's "hidden in the middle of
| the article", the note that the solution actually gets 72% on
| the subset of problems on which humans get ~85%. The fact that
| the claimed human baseline for ARC-AGI as a whole is based on
| an easy subset is pretty suspect.
| comfortabledoug wrote:
| I'm glad someone else finally said it, those born blind cannot
| possibly have AGI!
|
| /sarcasm :D
| nadam wrote:
| Amazing work, prompt engineering at its finest. One future
| direction for Arc AGI could be to use not Python, but a much more
| concise programming language that is more suited for brute-force
| methods like genetic mutations. The problem would be of course to
| train an LLM that is proficient enough in such a language. I am
| thinking about stack based languages. For this competition I
| would develop a careful bit-level encoding of a variant of the
| 'Joy' programming language.
| (https://en.wikipedia.org/wiki/Joy_(programming_language)) It
| would be a considerable effort though which I don't have time
| for, hence I post this idea publicly. A promising direction is a
| mix of things in my opinion: Special stack-based concise
| language, consulting LLMs like the OP did, and genetic algorithms
| combined.
| fire_lake wrote:
| S expressions are great for making valid syntax more likely -
| an old trick from Genetic Programming.
| YeGoblynQueenne wrote:
| >> Claim 1 seems likely true to me for a reasonable notion of
| "learning". I think Francois Chollet agrees here. Most of my
| doubts about this claim are concerns that you can basically brute
| force ARC-AGI without interestingly doing learning (e.g. brute-
| force search over some sort of DSL or training on a huge array of
| very similar problems). These concerns apply much less to the
| kind of approach I used
|
| The approach described in the article is exactly "brute-force
| search over some sort of DSL". The "DSL" is a model of Python
| syntax that GPT-4o has learned after training on the entire
| internet. This "DSL" is locked up in the black box of GPT-4o's
| weights, but just because no-one can see it, it doesn't mean it's
| not there; and we can see GPT-4o generating Python programs, so
| we know it is there, even if we don't know what it looks like.
|
| That DSL may not be "domain specific" in the sense of being
| specifically tailored to solve ARC-AGI tasks, or any other
| particular task, but it is "domain specific" in the sense of
| generating Python programs for some subset of all possible Python
| programs that includes programs that can solve some ARC-AGI
| tasks. That's a very broad category, but that's why it over-
| generates so much: it needs to draw 8k samples total until one
| works for just 50% of the public eval set.
| htrp wrote:
| The expectation is that you'll have to have dynamically generated
| benchmarks with better eval at some point given the potential for
| brute forcing the private validation set.
| sparsely wrote:
| You can have a go at the problems here:
| https://arcprize.org/play?task=00576224
|
| None of them are terribly hard but some aren't trivial either, a
| couple took me a bit of thinking to work out. By far the most
| tedious part is inputting the result (I didn't bother after the
| first) which is definitely something AI is better at!
| killerstorm wrote:
| FWIW GPT-4 is able to generate a plan very similar to one in the
| article: also involves feature extraction, program synthesis,
| iterative refinement.
|
| https://chatgpt.com/share/2fde1db5-00cf-404d-9ae5-192aa5ac90...
|
| So it's pretty close to being able to plan solution completely on
| its own. It's just rather bad at coding and visual inputs, so it
| doesn't know what it doesn't know.
| uptownfunk wrote:
| Arc agi is a small stepping stone to agi but is not agi.
|
| Program search mimics what humans do to a certain extent but not
| in entirety.
|
| A more general world model and reference will be required for
| agi.
| cchance wrote:
| LOL i looked at that first complex test sample and closed the
| page, it made my brain hurt.
| machiaweliczny wrote:
| This challenge looks quite solvable but it's relies on physics
| understanding and it's has a lot of human/world priors in sense
| of space understanding and object boundaries.
|
| Seems like it relies on identification of objects and then
| mapping them somehow. Most of the cases so far that I've seen are
| based on some transformation or relation between the objects.
|
| So far it seems like some search among common transformatiosn and
| relations could solve it. Plus some heuristics/computation for
| counting order, wholeness(boundary) or pattern.
|
| IMO it can be solved by search of programs that combine these +
| some LLM to guide heuristics most likely.
|
| The only hard one was applied noise or one testing understanding
| of "gravity".
|
| Did anyone test human baseline for this?
___________________________________________________________________
(page generated 2024-06-18 23:02 UTC)