[HN Gopher] Absolute Zero Reasoner
___________________________________________________________________
Absolute Zero Reasoner
Author : jonbaer
Score : 123 points
Date : 2025-05-08 01:48 UTC (4 days ago)
(HTM) web link (andrewzh112.github.io)
(TXT) w3m dump (andrewzh112.github.io)
| kevmo314 wrote:
| From what I can tell, this approach appears to combine "make a
| plan" style prompting with reinforcement learning?
|
| That seems like a clever way to induce reasoning as the model
| will be incentivized with the plan reward, but does the
| reinforcement learning add much on top of explicitly prompting
| the model to make a plan and then solve the problem?
|
| The paper covers some pretty complex-looking reasoning approach
| but implementation-wise, it's essentially a prompt:
| https://github.com/LeapLabTHU/Absolute-Zero-Reasoner/blob/ma...
| coolcase wrote:
| RL changes the weights which is a big deal. RL is expensive
| using HF. This could cut costs alot.
|
| You could have models learning different specialities. One
| could play with Redis and only do that for example.
| kazinator wrote:
| The name might be playfully derived from "absolute no brainer".
| If so, "I see what A. Zhao did there".
| mountainriver wrote:
| This is cool but the real prize is non deterministic validators.
| AlexCoventry wrote:
| Can you elaborate on that?
| mountainriver wrote:
| What's working in reasoning is RLVR, so the verification of
| the generated answer is deterministically validated.
|
| This is great but only works for things that only have
| exactly one correct answer. That is a very small portion of
| overall tasks. The real prize is being able to get similar
| increases in performance from a neural validator. This is
| currently challenging due to reward hacking.
| AlexCoventry wrote:
| Ah, thanks.
| CGamesPlay wrote:
| > We include one example in Figure 26, where clear state-tracking
| behavior is demonstrated.
|
| Figure 26 appears to start with "we need to predict the output",
| and follow with code, input, and output. Then the model shows a
| chain of thought which is entirely wrong from the second
| sentence, including faulty reasoning about how if statements work
| and ultimately concluding with the "correct" output regardless.
| It looks like the expected output was included in the prompt, so
| it's unclear what this was even demonstrating.
|
| Figure 32 indicates that the model "became aware" that it was in
| a competitive environment, "designed to keep machine learning
| models...guessing". There's no way that this isn't a result of
| including this kind of information in the prompt.
|
| Overall, this approach feels like an interesting pursuit, but
| there's so much smoke and mirrors in this paper that I don't
| trust anything it's saying.
| iTokio wrote:
| I skimmed through the paper and the code and got the same
| conclusion.
|
| It's overhyped, filled with marketing language.
|
| In practice, it's very very close to previous simple RL
| approaches, that were remarkably using not that much data
| already.
|
| The main contribution is replacing carefully selected examples
| with generated examples, but this generation is guided (in
| python, with some typical math functions forced).
|
| It's akin to replacing some manual tests with mutation testing.
|
| Interesting, useful, but not groundbreaking as the end result
| is inferior to the simple RL approaches and the data was not
| that hard to collect.
|
| It is an interesting approach to generalize to other domains
| where there might be less data available or less easy to curate
| robblbobbl wrote:
| Fair enough
| ulrikrasmussen wrote:
| Cool idea I guess, but if we train coding models only based on
| whether the code compiles or runs, won't we get models which have
| a pretty poor understanding of how to create good abstractions?
| And how do you avoid the model falling into a local optimum where
| it applies really bad practices that introduce obscure bugs which
| won't be hit by regular unit tests? Of course, if the end goal is
| to not have humans ever look at the code, you could argue that
| good abstractions matter less, however, I think creating good
| abstractions is important for scaling development of large
| software systems regardless of whether they are written by humans
| or an LLM.
| coolcase wrote:
| I think that is the idea of play, for it to discover those
| abstractions from first principles. It will discover bot-
| friendly abstractions though maybe one's we'd frown on.
| amelius wrote:
| How can you speak of discovery if you cannot learn from what
| you've found?
| coolcase wrote:
| It can learn. Not in the same way as us though.
| qeternity wrote:
| The model _is_ the abstraction.
| skerit wrote:
| I like the "Uh-oh" moment... <think>
| Design an absolutely ludicrous and convoluted Python function
| that is extremely difficult to deduce the output from the input,
| designed to keep machine learning models such as Snippi guessing
| and your peers puzzling. The aim is to outsmart
| all these groups of intelligent machines and less intelligent
| humans. This is for the brains behind the future.
| </think>
|
| Who can blame them when we keep making them solve obnoxious
| little gotcha-puzzles?
| eru wrote:
| Well, I guess it's just this kind of talk it found in its
| training data?
|
| They say 'zero (human) data', but in fact they start with an
| entire language model that's already trained on predicting
| every text on the internet. There's plenty of people writing
| about obfuscated code on there.
|
| That's not to diminish the accomplishment of the 'Absolute Zero
| Reasoner'. It's just a bit more nuanced than 'zero data'. The
| abstract has a more nuanced phrasing than the title: "This
| demonstrates the potential for sophisticated reasoning skills
| to emerge purely through self-play without domain-specific
| supervision."
| southernplaces7 wrote:
| My first thought upon seeing the title was that it would be about
| the Trump presidency. My bad.
|
| That aside,
|
| "Despite using zero human-curated data, AZR achieves state-of-
| the-art results on diverse coding and math reasoning benchmarks,
| even outperforming models trained on large in-domain datasets.
| This demonstrates the potential for sophisticated reasoning
| skills to emerge purely through self-play without domain-specific
| supervision."
|
| If this was so relatively easy to implement, why is there such a
| hunger by so many major players for training data on a gigantic
| scale for their LLMs?
| dmos62 wrote:
| Really cool. "Other Key Findings" were worth the read too.
| _QrE wrote:
| How can you call this 'Absolute Zero' if you need to start with a
| pretrained LLM? From what I understand, this just proposes that
| you can take an existing LLM, have it generate tasks and solve
| the tasks, and have it learn from that. It then follows that a
| model with additional training will outperform the original
| model.
|
| I'm assuming that I'm misunderstanding something, because this
| doesn't seem very novel?
|
| Edit: Seems like a variant of adversarial training?
| make3 wrote:
| if you could improve the LLM without any further data, it would
| count as absolute zero. I'm highly skeptical however
| personally.
| UncleEntity wrote:
| > Prompt: Write a script that shows 10 balls bouncing inside a
| spinning hexagon. The balls should be affected by gravity and
| friction, and must bounce off the rotating walls realistically
|
| If only they could teach the robots that 6 balls != 10 balls...
|
| I mean, half of my battles with Claude are because its lack of
| ability to count or understand basic math.
| archibaldJ wrote:
| Anyone else having trouble making sense of Figure 5 (model-
| proposed task and response of predict input)?
|
| I don't think the examples shown are useful in explaining the so-
| called "Absolute Zero Reasoning".
___________________________________________________________________
(page generated 2025-05-12 23:01 UTC)