[HN Gopher] HyperAgents: Self-referential self-improving agents
___________________________________________________________________
HyperAgents: Self-referential self-improving agents
https://arxiv.org/abs/2603.19461
Author : andyg_blog
Score : 228 points
Date : 2026-03-24 16:54 UTC (3 days ago)
(HTM) web link (github.com)
(TXT) w3m dump (github.com)
| flockonus wrote:
| The readme seems very unclear about what it does. Anyone has a
| practical example of it?
| pegasus wrote:
| There's a paper at https://arxiv.org/abs/2603.19461
|
| Abstract:
|
| Self-improving AI systems aim to reduce reliance on human
| engineering by learning to improve their own learning and
| problem-solving processes. Existing approaches to self-
| improvement rely on fixed, handcrafted meta-level mechanisms,
| fundamentally limiting how fast such systems can improve. The
| Darwin Godel Machine (DGM) demonstrates open-ended self-
| improvement in coding by repeatedly generating and evaluating
| self-modified variants. Because both evaluation and self-
| modification are coding tasks, gains in coding ability can
| translate into gains in self-improvement ability. However, this
| alignment does not generally hold beyond coding domains. We
| introduce \textbf{hyperagents}, self-referential agents that
| integrate a task agent (which solves the target task) and a
| meta agent (which modifies itself and the task agent) into a
| single editable program. Crucially, the meta-level modification
| procedure is itself editable, enabling metacognitive self-
| modification, improving not only the task-solving behavior, but
| also the mechanism that generates future improvements. We
| instantiate this framework by extending DGM to create DGM-
| Hyperagents (DGM-H), eliminating the assumption of domain-
| specific alignment between task performance and self-
| modification skill to potentially support self-accelerating
| progress on any computable task. Across diverse domains, the
| DGM-H improves performance over time and outperforms baselines
| without self-improvement or open-ended exploration, as well as
| prior self-improving systems. Furthermore, the DGM-H improves
| the process by which it generates new agents (e.g., persistent
| memory, performance tracking), and these meta-level
| improvements transfer across domains and accumulate across
| runs. DGM-Hyperagents offer a glimpse of open-ended AI systems
| that do not merely search for better solutions, but continually
| improve their search for how to improve.
| functional_dev wrote:
| This 'self vs non-self' logic is very similar to how plants
| prevent self-pollination. They have a biological
| 'discrimination' system to recognize and reject their own
| genetic code.
|
| Here is a breakdown - https://vectree.io/c/plant-self-
| incompatibility-logic
| OutThisLife wrote:
| Hermes agent does this, if you're curious
|
| https://github.com/NousResearch/hermes-agent
| willy_k wrote:
| Seems like that only has the task improvement loop, no self-
| improvement improvement loop like this project.
| jauntywundrkind wrote:
| Pi is self modifying, self aware.
| https://lucumr.pocoo.org/2026/1/31/pi/
|
| But this idea of having a task agent & meta agent maybe has
| wings. Neat submission.
| ghywertelling wrote:
| What are the differences wrt Recursive Language Models
| adw wrote:
| Completely unrelated. Recursive Language Models are just
| "what if we replaced putting all the long text into the
| context window with a REPL which lets you read parts of the
| context through tool calls and launch partitioned subagents",
| ie divide-and-conquer applied to attention space.
| bob1029 wrote:
| They also tend to imply symbolic recursion which seems to
| be the biggest deal out of everything by a wide margin.
|
| When you can nest 10+ agents deep and _guarantee_ you will
| get back home without losing any data in any of the stack
| frames, the ability to chunk through complex problems goes
| up dramatically.
| menaerus wrote:
| My first thought was also that this is also reminiscent of
| RLMs - they are ought to solve the same problem as far as
| my understanding goes. Authors say "Self-improving AI
| systems aim to reduce reliance on human engineering by
| learning to improve their own learning and problem-solving
| processes" which is what RLM is trying to solve so my
| understanding is that this work shares the same goal but
| takes a different approach. E.g. instead of using REPL-like
| environment with multiple (or even single) agents, which is
| what RLMs are doing, they suggest using agents that can
| modify themselves. I didn't read the paper so I don't know
| how this really works but it caught my attention so if you
| could share more insights I would appreciate it.
| ontouchstart wrote:
| I asked Pi to implement a skill. It was written in TS. Then I
| ask it to use the skill in two different sessions, none of them
| can get it working. One has to wrap it with JS and call it. The
| other has to take the curl commands out of the skill and call
| them directly. Which is quite smart BTW. But what is the point
| of making a convoluted TS skill at all?
|
| https://gist.github.com/ontouchstart/40db1244c0ff523f830df1c...
| Jerrrrrrrry wrote:
| No matter how far we go, we end up with generation /
| discrimination architecture.
|
| Its is the core of any and all learning/exellency; exposure to
| chaotic perturbations allow selection of solutions that are then
| generalized to further, ever more straining problems; producing
| increasingly applicable solutions.
|
| This is the core of evolution, and is actually derivable from
| just a single rule.
| ilaksh wrote:
| It's a feedback loop.
|
| I've always felt that the most important part of engineering
| was feedback loops.
|
| Maybe nature is the greatest engineer ever?
| 0xbadcafebee wrote:
| The most important part of engineering is problem-solving,
| which feedback loops don't necessarily do. The reason we are
| here as engineers is: 2.5 billion years ago, the earth made
| cyanobacteria, which flourished, then flooded the earth with
| toxic oxygen, killing almost all life on the planet. The
| initial feedback loop didn't solve a problem, it destroyed a
| use case. That's not a solution to a problem that an engineer
| would choose, even if those organisms that came after were
| pretty happy about it...
| Jerrrrrrrry wrote:
| Systems emerge in times of abundance, and are whittled in
| times of scarcity.
|
| The great oxygenation was a time of near catyclismsic
| scarcity for most complex organisms, as resources scale to
| food/energy requirements imply the most complex organisms
| were the most dependent on the environment, and were most
| impacted by changes.
|
| Inversely, oxygenation was our most crucial abundancy pre
| cursor, as it provides a large substrate chemically for
| life to exhibit
| Pausanias wrote:
| This process worked so spectacularly well that it
| eventually created human consciousness and the very concept
| of engineering... but I would never design a system that
| way because it killed version 1.0.
| NitpickLawyer wrote:
| Give hydrogen a few billion years and it starts wondering
| where it came from, and how it'd make v 2.0 better :)
| gobdovan wrote:
| I don't think generation/discrimination is fundamental. A more
| general framing is evolutionary epistemology (Donald T.
| Campbell, 1974, essay found in "The Philosophy of Karl
| Popper"), which holds that knowledge emerges through variation
| and selective retention. As Karl Popper put it, "We choose the
| theory which best holds its own in competition with other
| theories; the one which, by natural selection, proves itself
| the fittest to survive."
|
| On this view, learning in general operates via selection under
| uncertainty. This is less visible in individual cognition,
| where we tend to over-attribute agency, but it is explicit in
| science: hypotheses are proposed, subjected to tests, and
| selectively retained, precisely because the future cannot be
| deduced from the present.
|
| In that sense, generation/discrimination is a particular
| implementation of this broader principle (a way of
| instantiating variation and selection) not the primitive
| itself.
| Jerrrrrrrry wrote:
| I agree, I meant to be explicit that the one rule was
| "gravity";
|
| Variation (chaos) comes from the tidal push/pull of all
| cumulative processes - all processes are nearly periodic (2nd
| law) and get slower - guaranteeing oscillator harmonics at
| intervals.
|
| These intervals are astronomically convulted, but still
| promise a Fourier distribution of frequency: tidal effects
| ensure synchronization eventually, as all periods resonate
| eventually.
|
| As systems are increasingly exposed to pendulums of positive
| and negative coherence, they will generalize for variance,
| and eventually for increasingly (fourier) selective filters
| of increasingly resiliente traits, that will generalize.
|
| The system would eventually be increasingly resilient and
| eventually an awareness would develop.
|
| Awareness of past periodic cycles would improve fitness (with
| or without consciousness) and eventually the mechanistic
| processes would be in the systems nature.
|
| This is why we have pointless traditions, folk lore,
| collective unconscious artifacts, cyclical cataclysmic
| religions, the Fermi Paradox, the great filters...
|
| Variation and selection are woven, but understanding how it
| all stems from gravity by means of nearly perioidic
| oscillators (spinning planets, tidal pools, celestial bodies)
| due to the conservation of angular momentum, due to the 3body
| problem.....that is what took a genius to reconcile
| whattheheckheck wrote:
| So where does gravity come from?
| Jerrrrrrrry wrote:
| A cool illusion, just another emergent property of our
| geometrical solution: higher dimensional aperiodic
| tilings of a 10^80 faceted complex polyhedra "walking" on
| another large aperioidic Penrose plane, that is getting
| smaller in a dimension we observe as "energy".
|
| Basically a dice with a bajillion sides is getting rolled
| along an increasingly slim poker table, house winning
| eventually.
|
| Time only goes one way, protons dont decay, energy is
| radiated unto the cosmic background hiss, until
| homogeneity is reached as CMB, and entrophy reaches 1.
|
| I dont know where it comes from, but I know the shape it
| makes as it rolls by.
| zoogeny wrote:
| > eventually an awareness would develop
|
| I am not sure how this is a necessary conclusion to the
| premises you provide.
| Jerrrrrrrry wrote:
| Awareness would be any form of agency, goal seeking, or
| loss minimizing.
|
| As Briggs-Rauscher reactions can eventually lead to
| Belousov-Zhabotinsky reactions, the system can maintain
| homeostasis with its environment (and continuing to
| oscillate) by varying reactants in a loss minimizing
| fashion.
|
| This loss minimizing would be done during scarcity to
| limp towards an abundance phase.
|
| This is the mechanism that hypothetical tidal pools
| batteries would had exhibited to continue between periods
| of sunlight/darkness/acidity that eventually gets
| stratified as a resilency trait.
| zoogeny wrote:
| I'm not sure what your argument is here, except stating
| an opinion that loss minimization is equivalent to
| agency. But even if that was accepted, which is a huge
| stretch, it doesn't stretch all the way to awareness.
| Jerrrrrrrry wrote:
| It is, in context of its place in the cosmic scale.
|
| Loss minimizing to a few problems will generalize into
| abstraction, and a few solutions will develop.
|
| These systems with more generalizable resilency traits
| will encounter increasingly varied selective sieves.
|
| Systems that survive this seive will exhibit increasingly
| sophisticated, generalizable solutions to prevent loss of
| needed dependent reactions/resources.
|
| These solutions must exert influence to be effective;
| influencing the environment for its own benefit.
|
| As systems influence their environment, delineation of
| "self" and "environment" becomes a fundamental barrier.
|
| The system would prefer itself, or be outcompeted by a
| similar system that does.
|
| This layer of semi-life like material would form between
| sunlight and the oscillating reaction, and eventually
| envelope it, minimizing surface tension by means of a
| spherical cell like structure.
|
| Small stuff runs off of loss minimizing at a force level
| for its mechanistic affect; from covalent bonds to
| cellular ion transport, the path of lesser resistance is
| the fundamental forces.
|
| As systems become more complex, the minimizing is less
| directly attributable to the fundamental forces and
| becomes more of a Byzantine dependency/feedback network.
|
| This byzantine labyrinth of interactions is called
| biology.
|
| The delineation of self, the ego.
|
| At the highest levels, geopolitics. At the human level,
| mate suppression. Lowest level, energy conservation.
|
| All loss minimizing
| luma wrote:
| I'm not sure if you're familiar with the work from the
| lab of Mike Levin at Tufts but I'm betting you'll find it
| interesting if not. Here's a taste
| https://pmc.ncbi.nlm.nih.gov/articles/PMC6923654/
|
| While I disagree with your notion that this is explicity
| due to gravity, the rest of your argument seems to align
| with some of this lab's work. Learning can be
| demonstrated on scales as low as a few molecules, way
| below what we would normally call "life".
| NitpickLawyer wrote:
| The paper is here - https://arxiv.org/pdf/2603.19461
|
| This, IMO is the biggest insight into where we're at and where
| we're going:
|
| > Because both evaluation and self-modification are coding tasks,
| gains in coding ability can translate into gains in self-
| improvement ability.
|
| There's a thing that I've noticed early into LLMs: once they
| unlock one capability, you can use that capability to compose
| stuff and improve on other, related or not, capabilities. For
| example "reflexion" goes into coding - hey, this didn't work, let
| me try ... Then "tools". Then "reflxion" + "tools". And so on.
|
| You can get workflows that have individual parts that aren't so
| precise become better by composing them, and letting one
| component influence the other. Like e2e coding gets better by
| checking with "gof" tools (linters, compilers, etc). Then it gets
| even better by adding a coding review stage. Then it gets even
| better by adding a static analysis phase.
|
| Now we're seeing this all converge on "self improving" by
| combining "improving" components. And so on. This is really cool.
| binarymax wrote:
| I disagree that evaluation is always a coding task. Evaluation
| is scrutiny for the person who wants the thing. It's
| _subjective_. So, unless you're evaluating something purely
| objective, such as an algorithm, I don't see how a self
| contained, self "improving " agent accomplishes the
| subjectivity constraint - as by design you are leaving out the
| subject.
| ranyume wrote:
| In science there are ways to surface subjectivity (cannot be
| counted) into observable quantized phenomena. Take opinion
| polls for instance: "approval" of a political figure can mean
| many things and is subjective, but experts in the field make
| "approval" into a number through scientific methods. These
| methods are just an approximation and have many IFs, they're
| not perfect (and for presidential campaign analysis in
| particular they've been failing for reasons I won't clarify
| here), but they're useful nonetheless.
|
| Another thing that get quantized is video preferences to
| maximize engagement.
| NitpickLawyer wrote:
| Sure. There will always be subjective tasks where the person
| who asks for something needs to give feedback. But even there
| we could come up with ways to make it easier / faster /
| better ux. (one example I saw my frontend colleagues do is
| use a fast model to create 9 versions of a component, in a
| grid. And they "at a glance" decide which one is "better",
| and use that going forwards).
|
| OTOH, there's _loads_ you can do for evaluation before a
| human even sees the artifact. Things like does the site load,
| does it behave the same, did anything major change on the
| happy path, etc etc. There 's a recent-ish paper where
| instead of classic "LLM as a judge" they used LLMs to come up
| with rubrics, and other instances check original prompt +
| rubrics on a binary scale. Saw improvements in a lot of
| evaluations.
|
| Then there's "evaluate by having an agent do it" for any
| documentation tracking. Say you have a project, you implement
| a feature, and document the changes. Then you can have an
| agent take that documentation and "try it out". Should give
| you much faster feedback loops.
| alsetmusic wrote:
| > Things like does the site load, does it behave the same,
| did anything major change on the happy path, etc etc.
|
| I asked Claude to build a web app to run locally polling
| data from the LAN. It fought me for four rounds of me
| telling it that the data from the api wasn't rendered on
| the page. It created tests with mock data, it validated the
| api, it tested that the page loaded. It was gaslighting
| telling me that everything worked every time I told it that
| it didn't work. I had to tell it to inspect the dom and
| take screenshots with Playwright to make it stop effing
| around. I don't think it ever would have found the right
| response on its own.
|
| Even after deliberate intervention, it regressed a few
| rounds later and stopped caring that tests failed.
| Whatever, I don't treat it as anything more than a
| sometimes-correct random output machine.
| lukebuehler wrote:
| Agree. It's code all the way down. The key is to give agents a
| substrate where they can code up new capabilities and then
| compose them meaningfully and safely.
|
| Larger composition, though, starts to run into typical software
| design problems, like dependency graphs, shared state, how to
| upgrade, etc.
|
| I've been working on this front for over two years now too:
| https://github.com/smartcomputer-ai/agent-os/
| whattheheckheck wrote:
| So what are software packages now a days other than
| precomputed subsets of capabilities. Like a mesh that data
| gets pushed through to produce what? What are the optimal
| subset of prebuilt programs to accomplish any task?
| lukebuehler wrote:
| It's a tradeoff. Technically, you need very few programs,
| you can let an agent do everything and coordinate
| everything. But that is also inefficient, it's slow and
| uses a lot of tokens. So you allow the agent to build tools
| and coordinate those tools, just like we humans do.
| However, with agents, the threshold of pain is much higher,
| we can let agents do thing's "manually" where humans would
| build automations much sooner.
| whattheheckheck wrote:
| Oh wow, what do you think of karpathys autoresearch? Feels
| like this is just that? Gotta openclawify it?
| iamandoni wrote:
| > Larger composition, though, starts to run into typical
| software design problems
|
| I've been seeing the same thing. Where agents are great
| solving the immediate task, but as changes compound they run
| into software & architectural design problems. I created
| https://github.com/andonimichael/arxitect to help at least
| have coding agents self reflect on their software design. But
| I really like your approach to self-modification and
| improving the agent itself instead of just teaching it
| another skill in it's context.
| dataviz1000 wrote:
| I've been working with Claude Code to create copies of itself
| using git worktrees, run an iteration and then update its
| instructions. It can reverse engineer every website I tested
| it on. I kept updating the instructions then started asking
| Claude to update itself. Then asked if could figure out who
| to iterate unsupervised.
| https://github.com/adam-s/intercept?tab=readme-ov-
| file#the-s...
| testaccount28 wrote:
| because submarine piloting is a going-under-water activity,
| improvements in holding one's breath can lead to faster
| submersibles.
| alansaber wrote:
| The whole theme of llm dev to date has been "theres more common
| than not" in llm applications
| Atomic_Torrfisk wrote:
| Im sorry, this just sounds like hypespeak. CAn you provide
| samples?
|
| > once they unlock one capability,
|
| What does it mean to unlock? Its an llm nothing is locked. The
| output is a as good as the context, model and environment.
| Nothing is hidden or locked.
| IncreasePosts wrote:
| Maybe unlock means "recognize and solve a problem with an
| order of magnitude fewer tokens than the first time you did
| it". The same way humans might spend a lot of time thinking
| about a certain problem and various ways to solve it, but
| once they go through that process, and then recognize it
| again, they don't need to go to the same process and jump
| right to the solution.
| seertaak wrote:
| I'll have a stab at this. I'll start with an attempt at
| justifying the remark that an agent which is a good coder
| will be good at other tasks.
|
| 1. Coding is, as a technical endeavour, relatively difficult
| (similarly for mathematics). So a model which performs well
| on this task can be expected to easily handle also-technical-
| but-slightly-easier tasks, like understanding (musical)
| harmony theory or counterpoint -- for much the same reason
| that human programmers/mathematicians/scientist don't
| struggle to understand those "easier" theories.
|
| 2. Reinforcement learning augments a base models ability to
| excel in something else that's "difficult", namely to "look
| ahead" and plan multiple steps in advance. That's literally
| how the training algorithm works, generating multiple paths
| at once, and rewarding intermediate steps in those paths
| which succeed in attaining the goal. And that skill, too, is
| extremely useful in other domains. An AI agent which learns
| that to break a problem into sub-problems, and then tackle
| each in turn methodically -- it stands to reason that it can
| apply that to, say, a business plan.
|
| Note: 1 & 2 are not independent, nor are frontier models'
| excellence in these domains magical: it ultimately boils down
| to the availability of _massive datasets_ (in particular for
| coding) and _totally objective_ metrics (in the case of
| mathematics: solved math problems). That 's the key
| ingrediant for reinforcement learning to be so effective.
|
| So: the skills are transferrable because they're difficult,
| and require lots of planning. _That_ models are so good at
| them is a fluke, and in a parallel world where humans created
| git repo after git repo of business plans, it might be _that_
| which we lean on to teach a reinforcement learning algorithm
| how to "reason" and "plan".
|
| Now let's turn our attention to the "synergies" aspect, which
| I agree with. Let's say your agentic model, which is already
| excellent at reasoning and planning, acquires a new or
| improved capability which allows it to search the domain
| space, calculate, etc. much better than before -- this
| capability can now bear upon the plan, or be factored into
| the plan. For example, the model might be able to say "I
| don't need to worry about this particular subproblem for now;
| I can rely on my "mathematica" capability to deal with it
| when I absolutely need."
|
| Or to put it differently: monkeys, like humans, are able to
| use (rudimentary) tools. They'll take a rock, and use it to
| crack open a coconut (or whatever). But a human being, with
| far superior reasoning and planning abilities, takes _that_
| tool, and uses it to make an _even better tool_ -- and the
| result after many iterations of this process is civilization
| as we know it, while monkeys are still stuck trying to crack
| open nuts with rocks.
| alexpotato wrote:
| >You can get workflows that have individual parts that aren't
| so precise become better by composing them, and letting one
| component influence the other. Like e2e coding gets better by
| checking with "gof" tools (linters, compilers, etc). Then it
| gets even better by adding a coding review stage. Then it gets
| even better by adding a static analysis phase.
|
| This is the exact point I make whenever people say LLMs aren't
| deterministic and therefore not useful.
|
| Yes, they are "stochastic". But you can use them to write
| deterministic tools that create machine readable output that
| the LLM can use. As you mention, you keep building more of
| these tools and tying them together and then you have a
| deterministic "network" of "lego blocks" that you can run
| repeatably.
| sbinnee wrote:
| I guess this paper is part of ICML coming soon this June. I
| hope to see a lot of cool papers.
| naasking wrote:
| Agents need the ability to code but also to objectively and
| accurately evaluate whether changes resulted in real
| improvements. This requires skills with metrics and statistics.
| If they can make those reliable then self-improvement is
| basically assured, on a long enough timeline.
| derek1800 wrote:
| This is how hyperagents work. They Have the ability to
| measure improvement in both the meta agent and task agents.
| There approach requires task agents to tackle tasks that can
| be empirically evaluated.
| FrustratedMonky wrote:
| IF they are self modifying. Is there also a big risk, that they
| cause a bug, to dumb themselves down, break themselves. How do
| they get back? Are they able to restore a backup of themselves,
| if a self modification is bad.
|
| Or, are there two. One is modifying the other, observing
| results, before self applying.
| derek1800 wrote:
| Yes this is how hyperagents work. They keep track of their
| different attempts to see which results in success. It does
| require tasks that can be measured.
| ACCount37 wrote:
| Yep. Behavior composition. If you train an LLM to do A and to
| do B, separately, chances are, it'll be decent at A+B despite
| not being trained for the combination.
| llmslave wrote:
| I think even code bases will have self improving agents. Software
| is moving from just the product code, to the agent code that
| maintains the product. Engineering teams/companies that move in
| this direction will vastly out produce others.
|
| I've had to really shift how I think about building code bases,
| alot of logic can go into claude skills and sub agents. Requires
| essentially relearning software engineering
| _pdp_ wrote:
| We do this already but I bet this is not how people imagine it
| to be. There is still a review process to accept contributions.
| sonu27 wrote:
| Can someone add this to OpenClaw :)
| kordlessagain wrote:
| I wish someone would add it to Nemesis8.
| Sabinus wrote:
| My brother in Christ it's 2026, just ask an AI to do it for $10
| in API credits.
| measurablefunc wrote:
| That's great but how about UltraAgents: Meta-referential meta-
| improving self-referential hyperagents?
| 2001zhaozhao wrote:
| AGI-MegaAgent 5.7 Pro Ultra
| measurablefunc wrote:
| Somehow still financed w/ ads & ubiquitous surveillance.
| kordlessagain wrote:
| Uses LiteLLM. Lovely.
| clarionbell wrote:
| Pinned to 1.74.9, so not compromised.
| mifydev wrote:
| I've been experimenting with similar concept myself. The linter
| loop is the only thing that can keep the agent sane in my
| opinion, and if anyone can generalize bun+tsc loop to other
| tasks, this would finally be a way to trust LLMs output.
|
| I was annoyed at how Claude Code ignores my CLAUDE.md and skills,
| so I was looking for ways to expand type checking to them. So I
| wrote a wrapper on top of claude-agents-sdk that reads my
| CLAUDE.md and skills, and compiles them into rules - could be
| linter rules or custom checking scripts. Then it hooks up to all
| tools and runs the checks. The self improving part comes if some
| rule doesn't work: I run the tool with the session id in review
| mode, it proposes the fixes and improves the rule checkers. (not
| the md files) So it's kinda like vibe coding rules, definitely
| lowers the bar for me to maintain them. Repo:
| https://github.com/chebykinn/agent-ruler
| whattheheckheck wrote:
| You could try wes mckinneys roborev
| supermdguy wrote:
| It's surprising that this works so well considering that AI-
| generated AGENTS.md files have been shown to be not very useful.
| I think the key difference here is that the real-world experience
| helps the agent reach regions of its latent space that wouldn't
| occur naturally through autoregression.
|
| I wonder how much of the improvement is due to the agent actually
| learning _new things_ vs. reaching parts of its latent space that
| enable it to recall things it already knows. Did the agent come
| up with novel RL reward design protocols based on trial and
| error? Or did the tokens in the environment cause it to "act
| smarter"?
| agrishin wrote:
| I found that running an agent in ralph loop, showing it the agent
| text and saying "run this, if it fails - identify the reason, and
| modify the agent instructions to avoid this, acceptance criteria
| are this and that" worked surprisingly well. Not sure if it
| qualifies as a self-referential self improving, but it was
| something.
| NoToP wrote:
| "So, what do you see as your greatest weakness?"
| pjio wrote:
| Ethical constraints. Just let me fix this...
| kordlessagain wrote:
| The loop on this is basically tweak your prompt until you score
| better on a contrived test.
| fmbb wrote:
| Yeah if it was truly capable of self-improving, why did it not
| take over the world yet?
|
| Gemini itself says AGI will be here in 2029, with human level
| intelligence and self-improvement capabilities. But then it
| will take until 2045 before the singularity. I don't understand
| what they are going to do in all those years.
| galaxyLogic wrote:
| I think Singularity is hype. What does it mean? Machines do
| something we can not understand? So talking about Singularity
| is really talking about something we can not talk about
| because we don't undertand what we are talking about?
|
| Wittgenstein said "From what we can not speak of, we must be
| silent about". That sounds like a tautology but I think there
| is a deeper meaning behind it.
|
| It means simply that once you start talking about what we can
| not talk about, you are already talking about that and
| therefore it is NOT something you can NOT talk about. Clearly
| we can talk about it because we are already talking about it.
| And therefore it is not something that can not be talked
| about. That is a paradox, a bit like Godel's, but something
| that doesn't contradict itself.
| XorNot wrote:
| You got it in the third sentence and then dismissed it for
| some reason?
|
| That's exactly what the Singularity is: it's the transition
| point beyond which meaningful predictions aren't possible.
|
| In a black hole it's the center where relativity breaks
| down.
|
| In AI it's the point at which non-human intelligence no
| longer requires human intelligence for self improvement:
| after which predictions of the future become somewhat
| meaningless.
|
| In the human lives experience, I would argue its like
| having your first child: you can know what's coming, study
| the theory, know everything to expect and youre still you
| on the other side...but you can't really know what will
| happen till you get there.
| galaxyLogic wrote:
| Good definitions. Would you then agree that when we reach
| the point where AI can improve itself without our help,
| it is still possible to make predictions, about it?
|
| I think we are already in the stage where AI can and does
| improve itself. But why should this stage be called
| "Singularity"? Like a Black Hole? That sounds like hype
| to me.
|
| When AI can improve itself, wouldn't it still be able to
| explain to us how it has improved itself? If it can not
| it still has a lot of improvement to do.
|
| Or are we saying that some things are "unexplainable" and
| AI will discover such things without being able to
| explain to us what they are? That sounds like mysticism,
| or hype to me. Or religion. We can not explain God,
| right?
| Drakim wrote:
| I am capable of self-improving yet I haven't taken over the
| world yet.
| zer00eyz wrote:
| The above comment needs to be higher.
|
| IF we had a black box programing language, and handed it over
| to this system, it would never be able to do anything with it
| past its context window.
|
| Hey kids I hear you like agents, so we made an agent write
| agents till we got better agents.
| padolsey wrote:
| If true, this is essentially what Dspy does
| https://github.com/stanfordnlp/dspy (it was made back in 2023)
| yurimo wrote:
| Sigh, as someone who does research in this area, this paper and
| its promotion on X has so many hype terms it is almost off-
| putting. If you read the paper what they are doing is trying to
| modify the scaffolding around a frozen FM until they get
| something better. None of this obviously includes any training
| (change to weights) or the underlying architecture. Even for
| scaffolding, a lot is still human-scaffolded: the outer loop
| (parent selection, evaluation protocol, task distribution) is
| mostly fixed. They experimented with editing parent selection and
| it rediscovers heuristics like UCB/softmax, but doesn't yet beat
| handcrafted versions, so a lot of metrics are incremental, which
| is okay, that is what research is often. But it's not like a run
| away self-improvement or "improve forever" that people spin
| online.
|
| It is an extension of their DGM paper. Also it's ~88M+ tokens per
| full run I think, not surprising as any sort of exploratory
| search is expensive and I commend them for releasing the code
| online because it pushes this small subfield. But people need to
| temper their expectations. IMO the best part is a nice transfer
| between improvement objectives after exhaustive iteration that
| they found. I am wondering if what we have here is a way to
| exhaust local search space, by letting the model better express
| it.
|
| On a separate one thing I think a lot about is whether these
| unchecked hyped claims and terms and marketing of papers actually
| does more bad than good to the field by setting expectations that
| cannot be delivered and distracting from the actual hard and
| unsexy nature of problems that need to be solved.
| redanddead wrote:
| Do you notice a lack of creativity in AI research today? What's
| your take
| neuracerebra-AI wrote:
| holy crap
| gcanyon wrote:
| We're going to find that the arc of self-optimization doesn't
| (didn't?) point toward success, right up until it does. And at
| that point agents, code, etc. are going to explode.
| JStanton617 wrote:
| At long last we've created Wintermute from William Gibson's
| classic 1984 novel Don't Create Wintermute
| grahammccain wrote:
| Being able to use new tools in ways we didn't think of has been a
| great part of my experience with Claude.
| sva_ wrote:
| It appears like the 'self-improving' here just means modifying
| the agent's prompt/context? And not actually changing any of the
| weights/architecture of a model. I feel like this kind of self-
| improvement has some hard limits on how much it can improve.
| internet101010 wrote:
| Definitely isn't perfect and has limitations, but if the goal
| of predictable outcomes in a dynamic environment at scale it's
| more feasible than creating fine tuned models for every little
| thing and allows for context-based model performance
| benchmarking.
| georaa wrote:
| How do you run self-improving agents in production though? Agent
| OOM-kills mid-improvement - state gone. Agent spawns sub-agents -
| no idea if they finished or died. Agent needs human sign-off -
| can't just block on stdin. I build multi-agent infra and 90% of
| the work is this boring stuff. Checkpointing, delivery tracking,
| async human oversight. The agent logic itself is maybe 10%.
___________________________________________________________________
(page generated 2026-03-27 23:01 UTC)