[HN Gopher] Learnings from building AI agents
___________________________________________________________________
Learnings from building AI agents
Author : pomarie
Score : 151 points
Date : 2025-06-26 12:45 UTC (10 hours ago)
(HTM) web link (www.cubic.dev)
(TXT) w3m dump (www.cubic.dev)
| bumbledraven wrote:
| What model were they using?
| jangletown wrote:
| "51% fewer false positives", how were you measuring? is this an
| internal or benchmarking dataset?
| N_Lens wrote:
| Very vague post light on details, and as usual, feels more like a
| marketing pitch for the website.
| flippyhead wrote:
| I found it useful.
| weego wrote:
| It's recreating the monolith vs micro-service argument by proxy
| for a new generation to plan conference talks around.
| vinnymac wrote:
| I've been testing this for the last few months, and it is now
| much quieter than before, and even more useful.
| kurtis_reed wrote:
| There was a blog post from another AI code review tool: "How to
| Make LLMs Shut Up"
|
| https://news.ycombinator.com/item?id=42451968
| h1fra wrote:
| what I saw using 5-6 tools like this:
|
| - PR description is never useful they barely summarize the file
| changes
|
| - 90% of comments are wrong or irrelevant wether it's because
| it's missing context, missing tribal knowledge, missing code
| quality rules or wrongly interpret the code change
|
| - 5-10% of the time it actually spots something
|
| Not entirely sure it's worth the noise
| bwfan123 wrote:
| code-reviews are not a good use-case for LLMs. here's why: LLMs
| shine in usecases when their output is not evaluated on
| accuracy - for example, recommendations, semantic-search,
| sample snippets, images of people riding horses etc. code-
| reviews require accuracy.
|
| What is a useful agent in the context of code-reviews in a
| large codebase is a semantic search agent which adds a comment
| containing related issues or PRs from the past for more context
| to human reviewers. This is a recommendation and is not rated
| on accuracy.
| asdev wrote:
| the code reviews can't be effective because the LLM does not
| have the tribal knowledge and product context of the change.
| it's just reading the code at face value
| mosura wrote:
| Lessons.
| chanux wrote:
| https://nolearnings.com/
| flippyhead wrote:
| This is LITERALLY mind blowing.
| criddell wrote:
| I don't like the word _learnings_ either, but you write for
| your audience and this article was probably written with the
| hope that it would be shared on LinkedIn.
|
| _Learnings_ might be the right choice here.
|
| I wouldn't complain if the HN headline mutator were to
| replace "Learnings" with "lessons".
| curiousgal wrote:
| > _Encouraged structured thinking by forcing the AI to justify
| its findings first, significantly reducing arbitrary
| conclusions._
|
| Ah yes, because we know very well that the current generation of
| AI models reasons and draws conclusions based on logic and
| understanding... This is the true face palm.
| nico wrote:
| Humans work pretty much the same way
|
| Several studies have shown that we first make the decision and
| then we reason about it to justify it
|
| In that sense, we are not much more rational than an LLM
| disgruntledphd2 wrote:
| Humans have a _lot_ more introspection capabilities than any
| current LLM.
| alganet wrote:
| > Several studies
|
| Please, cite those studies. I want to read them.
| elzbardico wrote:
| The "confidence" field in the structured output was what really
| baffled me.
| nzach wrote:
| I agree with the sentiment of this post. I my personal experience
| the usefulness of a LLM positively correlated with your ability
| to constrain the problem it should solve.
|
| Prompts like 'Update this regex to match this new pattern'
| generally give better results than 'Fix this routing error in my
| server'.
|
| Although this pattern seems true empirically, I've never seen any
| hard data to confirm this property(?). And this post is
| interesting but seems like a missed opportunity to back this idea
| with some numbers.
| exitb wrote:
| This seems like really bad news for the ,,AI will soon replace
| all software developers" crowd.
| singron wrote:
| I think they skipped over a non-obvious motivating example too
| fast. On first glance, commenting out your CI test suite would be
| very bad to sneak into a random PR, and that review note might be
| justified.
|
| I could imagine the situation might actually be more nuanced
| (e.g. adding new tests and some of them are commented out), but
| there isn't enough context to really determine that, and even in
| that case, it can be worth asking about commented out code in
| case the author left it that way by accident.
|
| Aren't there plenty of more obvious nitpicks to highlight? A
| great nitpick example would be one where the model will also ask
| to reverse the resolution. E.g. final var items
| = List.copyOf(...); <-- Consider using an explicit type
| for the variable. final List items =
| List.copyOf(...); <-- Consider using var to avoid
| redundant type name.
|
| This is clearly aggravating since it will always make review
| comments.
| willsmith72 wrote:
| yep completely agreed, how can that be the best example they
| chose to use?
|
| If I reviewed that PR, absolutely I'd question why you're
| commenting that out. There better be a very good reason, or
| even a link to a ticket with a clear deadline of when it can be
| cleaned up/reverted
| mattas wrote:
| "After extensive trial-and-error..."
|
| IMO, this is the difference between building deterministic
| software and non-deterministic software (like an AI agent). It
| often boils down to randomly making tweaks and evaluating the
| outcome of those tweaks.
| s1mplicissimus wrote:
| Afaik alchemists had a more reliable method than ... whatever
| this state of affairs is ^^
| snapcaster wrote:
| You're saying alchemy is better than the scientific method?
| AndrewKemendo wrote:
| Otherwise known as science
|
| 1:Observation 2:Hypothesis 3:test 4:GOTO:1
|
| This is every thing ever built ever
|
| What is the problem exactly?
| wrs wrote:
| For one thing, what you learned can stop working when you
| switch to a new model, or just a newer version of the "same"
| model.
| neuronic wrote:
| That's because there is no intelligence or understanding
| involved. They are just trying to brute force a tool for a
| different purpose into their use case because marketing can't
| stop overselling AI.
| nico wrote:
| > 2.3 Specialized Micro-Agents Over Generalized Rules Initially,
| our instinct was to continuously add more rules into a single
| large prompt to handle edge cases
|
| This has been my experience as well. However, it seems like the
| platforms like Cursor/Lovable/v0/et al are doing things
| differently
|
| For example, this is Lovable's leaked system prompt, 1550 lines:
| https://github.com/x1xhlol/system-prompts-and-models-of-ai-t...
|
| Is there a trick to making gigantic system prompts work well?
| shenberg wrote:
| When I read "51% fewer false positives" followed immediately by
| "Median comments per pull request cut by half" it makes me wonder
| how many true positives they find. That's maybe unfair as my
| reference is automated tooling in the security world, where the
| true-positive/false-positive ratio is so bad that a 50% reduction
| in false positives is a drop in the bucket
| Oras wrote:
| The problem is that, regardless of how you try to use "micro-
| agents " as a marketing term, LLMs are instructed to return a
| result.
|
| They will always try to come up with something.
|
| The example provided was a poor one. The comment from LLM was
| solid. Why would you comment out a step in the pipeline instead
| of just deleting it? I would comment the same in a PR.
| SparkyMcUnicorn wrote:
| I've found that giving agents an "opt out" works pretty well.
|
| For structured outputs, making fields optional isn't usually
| enough. Providing an additional field for it to dump some
| output, along with a description for how/when it should be
| used, covers several issues around this problem.
|
| I'm not claiming this would solve the specific issues discussed
| in the post. Just a potentially helpful tip for others out
| there.
| bjorgen wrote:
| Do you have an example of this in practice? I'm having a hard
| understanding this and have a very similar problem of the
| agent wanting to give a response on optional fields.
| ffsm8 wrote:
| Likely because it's temporary?
|
| It takes less effort to re-enable if it's just commented out
| and its more visible that there is something funky going on
| that _someone_ should fix.
|
| But yeah, even if it's temporary, it really should have the
| rationale for commenting it out added... It takes like 5s and
| provides important context for reviewers and people looking
| through the file history in the future.
| elzbardico wrote:
| Funny thing is the structured output in the last example.
|
| ``` { "reasoning": "`cfg` can be nil on line 42; dereferenced
| without check on line 47", "finding": "Possible nil-pointer
| dereference", "confidence": 0.81 } ```
|
| You know the confidence value is completely bogus, don't you?
| sharkjacobs wrote:
| Do you mean that there is no correlation between confidence and
| false positives or other errors?
| ramity wrote:
| elzbardico is pointing out how the author is having the
| confidence value generated in the output of the response
| rather than it being the confidence of the output.
| bckr wrote:
| Is there research solid knowledge on this?
| baby wrote:
| this trick is being used by many apps (including Github
| copilot reviews). The way I see it, is that if the agent
| has an eager-to-please problem, then you give it a way
| out
| ramity wrote:
| I too once fell into the trap of having an LLM generate a
| confidence value in a response. This is a very genuine concern
| to raise.
| munificent wrote:
| Easy fix, just have the LLM generate: {
| "reasoning": "`cfg` can be nil on line 42; dereferenced without
| check on line 47", "finding": "Possible nil-pointer
| dereference", "confidence": 0.81,
| "confidence_in_confidence_rating": 0.54,
| "confidence_in_confidence_rating_in_confidence_rating": 0.12,
| "confidence_in_confidence_rating_in_confidence_rating_in_confid
| ence_rating": 0.98, // Etc... }
| zengid wrote:
| confidence all the way down
| GardenLetter27 wrote:
| Confidence is all you need.
| lgas wrote:
| True in many situations in life.
| volkk wrote:
| i immediately noticed the same thing, but to be fair, we don't
| know if it's enriched by a separate service that checks the
| response and uses some heuristics to compute that value. If
| not, yeah, that is an entirely made up and useless value
| MattSayar wrote:
| Could you have a higher-order reasoning LLM generate a better
| confidence rating? That's how eval frameworks generally work
| today
| skipants wrote:
| When I was younger and more into music, when I went to a
| concert I would often judge if a drummer was "good" based on if
| they were better than me or not. I knew enough about drumming
| to tell how good someone was at the different parts of having
| that skill but also knew enough to know that I was not even
| close to having what it took to be a professional drummer.
|
| This is what I feel like with this blogpost. I've barely
| scratched the surface of the innards of LLMs but even I know it
| should be completely obvious to anyone that has a product built
| around it that these confidence levels are completely made up.
|
| I've never heard or used cubic before today but that part of
| the blog post, along with the obvious LLM generated quality of
| it, gives a terrible first impression.
| baby wrote:
| you know everything is made up right? And yet it just works. I
| too use a confidence score in an bug finder app, Github seems
| to use them in copilot reviews, people will use them until it
| is shown not to work anymore.
|
| on the other hand this post https://www.greptile.com/blog/make-
| llms-shut-up says that it didn't work in their case:
|
| > Sadly, this also failed. The LLMs judgment of its own output
| was nearly random. This also made the bot extremely slow
| because there was now a whole new inference call in the
| workflow.
| jstummbillig wrote:
| The multi agent thing with different roles is so obviously not a
| great concept, that I am very hesitant to build towards it, even
| thought it seems to win out right now. We want a AI that
| internally does what it needs to do to solve a problem, given a
| good enough problem description, tools and context. I really do
| not want to have to worry about breaking up tasks into chunks
| that are smaller than what I could handle myself, and I really
| hope that that in the near future this will go away.
| brabel wrote:
| People creating products need to do what gives results right
| now. And I can attest that breaking up jobs into small steps
| seems to work better for most scenarios. When that becomes
| unnecessary, creating products that are useful will become much
| easier for sure, but I wouldn't hold my breath.
| bckr wrote:
| I'm not being sarcastic when I say that I think supervisor
| agents and agent swarms in general are the way forward here
| b0a04gl wrote:
| we tried something simple. suprisingly exposed a lot; just ran
| same input twice through the agent, temp 0. diffed the reasoning
| trace token by token, didn't expect much honestly. but even small
| shifts showed up. one run said 'this may introduce risk'. other
| said 'this could cause issues'.. exact same code. made us realise
| prompt wasn't grounding the rationale path tight enough. wasn't
| hallucinating. just the why kept wobbling
| EnPissant wrote:
| > Explicit reasoning improves clarity. Require your AI to clearly
| explain its rationale first--this boosts accuracy and simplifies
| debugging.
|
| I wonder what models they are using because reasoning models do
| this by default, even if they don't give you that output.
|
| This post reads more like a marketing blog post than any real
| world advice.
| iandanforth wrote:
| I learned from a recent post
| (https://sean.heelan.io/2025/05/22/how-i-used-o3-to-find-cve-...)
| that finding security issues can take 100+ calls to an LLM to get
| good signal. So I wonder about agent implementers who are trying
| to get good signal out of single calls, even if they are
| specialized ones.
| bckr wrote:
| I think that article is talking about finding a previously
| unknown exploit. A known and well documented vulnerability
| should be much easier to identify
| OnionBlender wrote:
| What's funny about the bullet points in section 3 is that it only
| compares to the previous noisy agent, rather than having no
| agent. 51% fewer false positives, median comments per pull
| request cut by half, spending less time managing irrelevant
| comments? Turn it off and you could get a 100% reduction in false
| positives and spend zero time on irrevant AI generated comments.
___________________________________________________________________
(page generated 2025-06-26 23:01 UTC)