[HN Gopher] Project Glasswing: Securing critical software for th...
___________________________________________________________________
Project Glasswing: Securing critical software for the AI era
Related: _Assessing Claude Mythos Preview 's cybersecurity
capabilities_ - https://news.ycombinator.com/item?id=47679155
_System Card: Claude Mythos Preview [pdf]_ -
https://news.ycombinator.com/item?id=47679258 Also: _Anthropic 's
Project Glasswing sounds necessary to me_ -
https://news.ycombinator.com/item?id=47681241
Author : Ryan5453
Score : 1474 points
Date : 2026-04-07 18:09 UTC (1 days ago)
(HTM) web link (www.anthropic.com)
(TXT) w3m dump (www.anthropic.com)
| Ryan5453 wrote:
| Pricing for Mythos Preview is $25/$125, so cheaper than GPT 4.5
| ($75/$150) and GPT 5.4 Pro ($30/$180)
| cassianoleal wrote:
| Where did you get that from?
|
| From TFA:
|
| > We do not plan to make Claude Mythos Preview generally
| available
| Tiberium wrote:
| From the article:
|
| > Anthropic's commitment of $100M in model usage credits to
| Project Glasswing and additional participants will cover
| substantial usage throughout this research preview.
| Afterward, Claude Mythos Preview will be available to
| participants at $25/$125 per million input/output tokens
| (participants can access the model on the Claude API, Amazon
| Bedrock, Google Cloud's Vertex AI, and Microsoft Foundry).
| underdeserver wrote:
| Key point: available _to participants_.
| conradkay wrote:
| permanent underclass has arrived :(
| philipwhiuk wrote:
| give it a couple months
| conradkay wrote:
| For comparison, 5x the cost of Opus 4.6, and 1.67x for Opus 4.1
|
| I think this would be _very_ heavily used if they released it,
| completely unlike GPT 4.5
| adi_kurian wrote:
| Opus 4 & 4.1 are still on Vertex+Bedrock @ $75/1mm out. They
| were used very heavily and in my subjective opinion are
| better than 4.5 and 4.6.
| breakingcups wrote:
| Interesting, what makes them better to you?
| adi_kurian wrote:
| Opus 4, with enough context, could do most all I wanted
| in a single shot. More often than not, when I had a bad
| outcome and was frustrated I would realize that I was the
| problem (in giving improper direction or missing key
| context).
|
| I also was in a pretty sweet position having a boat load
| of credits and premo vertex rate limits so I could
| 'afford' to dump hundreds of thousands of tokens in
| context all day.
|
| With Opus 4.5 and 4.6, I find I have to steer very
| actively.
|
| This is comparing using Opus 4 directly rather than
| comparing the performance of the models in Claude Code
| for example, or any 'agentic' setup.
|
| Kinda reminds me of 4o vs 4-turbo.
|
| I would imagine they are smaller models.
| LoganDark wrote:
| It's nice to know that they continue to be committed to
| advertising how safe and ethical they are.
| rvz wrote:
| They are not our friends and _are_ the exact opposite of what
| they are preaching to be.
|
| Let alone their CEO scare mongering and actively attempting to
| get the government to ban local AI models running on your
| machine.
| simianwords wrote:
| How would you expect them to behave if they were your
| friends?
| ethin wrote:
| IMO (not the GP) but if Anthropic were my friends I would
| expect them to publish research that didn't just inflate
| the company itself and that was both reproduceable and
| verifiable. Not just puff pieces that describe how ethical
| they are. After all, if a company has to remind you in
| every PR piece that they are ethical and safety-focused,
| there is a decent probability that they are the exact
| opposite.
| Miraste wrote:
| They are a for-profit company, working on a project to
| eliminate all human labor and take the gains for
| themselves, with no plan to allow for the survival of
| anyone who works for a living. They're definitionally not
| your friends. While they remain for-profit, their specific
| behaviors don't really matter.
| simianwords wrote:
| I work for a tech company that eliminates a form of human
| labour and they remain for profit
| Miraste wrote:
| Sure, most tech companies eliminate some form of human
| labor. Anthropic aims to eliminate _all_ human labor,
| which is very different.
| SilverElfin wrote:
| I agree attempting to ban local AI models or censor them, is
| not appropriate. At the same time, they do seem far more
| ethical and less dangerous than other AI companies. And I
| include big tech in that - a bunch of greedy companies that
| just want to abuse their monopoli ... I mean moats.
| raldi wrote:
| In what ways is Anthropic different from a hypothetical
| frontier lab that you would characterize as legitimately safe
| and ethical?
| LoganDark wrote:
| I'm just a little frustrated they keep going on about how
| safe and ethical they are for keeping the more advanced
| capabilities from us. I wish they would wait to make an
| announcement until they have something to show, rather than
| this constant almost gloating.
| 0x3f wrote:
| Its existence is possible.
| ehutch79 wrote:
| Just include 'make it secure' in the prompt. Duh.
|
| /s
| zachperkel wrote:
| Mythos Preview has already found thousands of high-severity
| vulnerabilities, including some in every major operating system
| and web browser.
|
| Scary but also cool
| fsflover wrote:
| Every piece of software definitely has serious vulnerabilities,
| perfection is not achievable. Fortunately we have another
| approach to security: security through compartmentalization.
| See: https://qubes-os.org
| syndeo wrote:
| Once you get the compartmentalization working well, and "all"
| of the vulnerabilities are out of it too, of course...
|
| But even then you'll have users putting things in the same
| compartment for convenience, rather than leaving them
| properly sequestered.
| fsflover wrote:
| > and "all" of the vulnerabilities are out of it too
|
| This is a good point; however the isolating code should be
| much smaller and easier to verify.
| dakolli wrote:
| Or more likely, its just an exaggeration or lie.
| solenoid0937 wrote:
| Yes I'm sure this is all a massive conspiracy by the many
| companies that are making statements alongside Anthropic
| pertymcpert wrote:
| What evidence makes you say that? Do you have insider info?
| nicce wrote:
| Neither party provided the evidence. I wonder why people
| like to take the side of the optimistic.
| stratos123 wrote:
| We already know Opus can find real vulnerabilities ([1],
| [2], ...), so it's not exactly surprising that a bigger
| model is better at it.
|
| [1] https://news.ycombinator.com/item?id=47273854
|
| [2] https://news.ycombinator.com/item?id=47611921
| nicce wrote:
| That is not thousands high-severity vulnerabilities as
| above commenter stated. Even many local models have found
| individual vulnerabilities.
| ex-aws-dude wrote:
| Did someone actually go through all of those and check if they
| are high-severity or did the AI just tell them that?
| DiffTheEnder wrote:
| They mention that they have humans review the most crticial
| bugs before sending it to the maintainers in their dev blog.
| jryio wrote:
| Let's fast forward the clock. Does software security converge on
| a world with fewer vulnerabilities or more? I'm not sure it
| converges equally in all places.
|
| My understanding is that the pre-AI distribution of software
| quality (and vulnerabilities) will be massively exaggerated. More
| small vulnerable projects and fewer large vulnerable ones.
|
| It seems that large technology and infrastructure companies will
| be able to defend themselves by preempting token expenditure to
| catch vulnerabilities while the rest of the market is left with a
| "large token spend or get hacked" dilemma.
| mlinsey wrote:
| I'm pretty optimistic that not only does this clean up a lot of
| vulns in old code, but applying this level of scrutiny becomes
| a mandatory part of the vibecoding-toolchain.
|
| The biggest issue is legacy systems that are difficult to patch
| in practice.
| pipo234 wrote:
| Wait. Wasn't AI supposed to alleviate the burden of legacy
| code?!
| mlinsey wrote:
| If we have the source and it's easy to test, validate, and
| deploy an update - AI should make those easier to update.
|
| I am thinking of situations where one of those aren't true
| - where testing a proposed update is expensive or
| complicated, that are in systems that are hard to
| physically push updates to (think embedded systems) etc
| rattlesnakedave wrote:
| Legacy code, not the running systems powered by legacy code
| buzzerbetrayed wrote:
| If you're still an AI skeptic at this point, I don't know
| what sort of advancement could convince you that this is
| happening.
| wslh wrote:
| I imagine that some levels of patching would be improving as
| well, even as a separate endeavor. This is not to say that
| legacy systems could be completely rewritten.
| qingcharles wrote:
| I could see some of these corps now being able to issue more
| patches for old versions of software if they don't have to
| redirect their key devs onto prior code (which devs hate). As
| you say though, in practice it is hard to get those patches
| onto older devices.
|
| I'm looking at you, Android phone makers with 18 months of
| updates.
| phist_mcgee wrote:
| Yeah but who pays the enormous cost?
| khalic wrote:
| obviously the people responsible for the software. Would
| you rather anthropic kept the vulns quiet?
| repelsteeltje wrote:
| Off course not, but there is infinitely more vulnerable
| software escaping Anthropic's scrutiny. And when AI-
| powered discovery becomes a necessity, that will lead to
| concentration of power to these kinds of companies.
|
| Bruce Scheier made a comprehensive analysis of the pros
| and cons and forces at play for adversary and defenders
| [1].
|
| I think it's safe to predict yet more money previously
| directed to us techies will find its way to the
| Anthropics of this world.
|
| [1] https://www.schneier.com/blog/archives/2026/04/cybers
| ecurity...
| timschmidt wrote:
| Most vulnerabilities seem to be in C/C++ code, or web things
| like XSS, unsanitized input, leaky APIs, etc.
|
| Perhaps a chunk of that token spend will be porting legacy
| codebases to memory safe languages. And fewer tokens will be
| required to maintain the improved security.
| torginus wrote:
| I think most vulnerabilities are in crappy enterprise
| software. TOCTOU stuff in the crappy microservice cloud app
| handling patient records at your hospital, shitty auth at a
| webshop, that sort of stuff.
|
| A lot of these stuff is vulnerable by design - customer
| wanted a feature, but engineering couldnt make it work
| securely with the current architecture - so they opened a
| tiny hole here and there, hopefully nobody will notice it,
| and everyone went home when the clock struck 5.
|
| I'm sure most of us know about these kinds of vulnerabilities
| (and the culture that produces them).
|
| Before LLMs, people needed to invest time and effort into
| hacking these. But now, you can just build an automated vuln
| scanner and scan half the internet provided you have enough
| compute.
|
| I think there will be major SHTF situations coming from this.
| timschmidt wrote:
| Yeah. Crufty cobbled together enterprise stuff will suffer
| some of the worst. But this will be a great opportunity for
| the enterprise software services economy! lol.
|
| I honestly see some sort of automated whole codebase
| auditing and refactoring being the next big milestone along
| the chatbot -> claude code / codex / aider -> multi-agent
| frameworks line of development. If one of the big AI corps
| cracks that problem then all this goes away with the click
| of a button and exchange of some silver.
| pants2 wrote:
| Software security heavily favors the defenders (ex. it's much
| easier to encrypt a file than break the encryption). Thus with
| better tools and ample time to reach steady-state, we would
| expect software to become more secure.
| justincormack wrote:
| Software security heavily favours the attacker (ex. its much
| easier to find a single vulnerability than to patch every
| vulnerability). Thus with better tools and ample time to
| reach steady-state, we would expect software to remain
| insecure.
| fsflover wrote:
| This is only true if your approach is security through
| correctness. This never works in practice. Try security
| through compartmentalization. Qubes OS provides it
| reasonably well.
| pants2 wrote:
| If we think in the context of LLMs, why is it easier to
| find a single vulnerability than to patch every
| vulnerability? If the defender and the attacker are using
| the same LLM, the defender will run "find a critical
| vulnerability in my software" until it comes up empty and
| then the attacker will find nothing.
|
| Defenders are favored here too, especially for closed-
| source applications where the defender's LLM has access to
| all the source code while the attacker's LLM doesn't.
| dist-epoch wrote:
| You also need to deploy the patch. And a lot of software
| doesn't have easy update mechanisms.
|
| A fix in the latest Linux kernel is meaningless if you
| are still running Ubuntu 20.
| Tepix wrote:
| It's not because fixes get backported.
| conradkay wrote:
| That generally makes sense to me, but I wonder if it's
| different when the attacker and defender are using the same
| tool (Mythos in this case)
|
| Maybe you just spend more on tokens by some factor than the
| attackers do combined, and end up _mostly_ okay. Put
| another way, if there 's 20 vulnerabilities that Mythos is
| capable of finding, maybe it's reasonable to find all of
| them?
| conradkay wrote:
| From the red team post
| https://red.anthropic.com/2026/mythos-preview/
|
| "Most security tooling has historically benefitted
| defenders more than attackers. When the first software
| fuzzers were deployed at large scale, there were concerns
| they might enable attackers to identify vulnerabilities
| at an increased rate. And they did. But modern fuzzers
| like AFL are now a critical component of the security
| ecosystem: projects like OSS-Fuzz dedicate significant
| resources to help secure key open source software.
|
| We believe the same will hold true here too--eventually.
| Once the security landscape has reached a new
| equilibrium, we believe that powerful language models
| will benefit defenders more than attackers, increasing
| the overall security of the software ecosystem. The
| advantage will belong to the side that can get the most
| out of these tools. In the short term, this could be
| attackers, if frontier labs aren't careful about how they
| release these models. In the long term, we expect it will
| be defenders who will more efficiently direct resources
| and use these models to fix bugs before new code ever
| ships. "
| intended wrote:
| This came across as so confident that I had a moment of
| doubt.
|
| It is most definitely an attackers world: most of us are
| safe, not because of the strength of our defenses but the
| disinterest of our attackers.
| Herring wrote:
| There are plenty of interested attackers who would love to
| control every device. One is in the white house, for
| example.
| tptacek wrote:
| I don't think this is broadly true and to the extent it's
| true for cryptographic software, it's only relatively
| recently become true; in the 2000s and 2010s, if I was tasked
| with assessing software that "encrypted a file" (or more
| likely some kind of "message"), my bet would be on finding a
| game-over flaw in that.
| lilytweed wrote:
| I think we're starting to glimpse the world in which those
| individuals or organizations who pigheadedly want to avoid
| using AI at all costs will see their vulnerabilities brutally
| exploited.
| woeirua wrote:
| Yep, it's this. The laggards are going to get brutally
| eviscerated. Any system connected to the internet is going to
| be exploited over the next year unless security is taken very
| seriously.
| skejeke wrote:
| lol and what about the vibe coders?
|
| You people are comical. Why do you feel the need to create
| so much hype around what you say? Did you not get enough
| attention as a kid?
| woeirua wrote:
| The vibe coders will be fine. They'll use LLMs to red
| team their code.
| skejeke wrote:
| What a load of nonsense.
| ofjcihen wrote:
| This was true before LLMs though.
|
| I mean even going back to Sasser.
| RALaBarge wrote:
| Botnet city: where everyone's a botnet, and the DDoS dont
| matter!
| cyanydeez wrote:
| I'm more curious as to just how fancy we can make our honey
| pots. These bots arn't really subtle about it; they're used as
| a kludge to do anything the user wants. They make tons of
| mistakes on their way to their goals, so this is definitely not
| any kind of stealthy thing.
|
| I think this entire post is just an advertisement to goad CISOs
| to buy $package$ to try out.
| rachel_rig wrote:
| You'd think they would have used this model to clean up
| Claude's own outage issues and security issues. Doesn't give me
| a lot of faith.
| tdaltonc wrote:
| Depends - do you think people are good at keeping their fridge
| firmware up-to-date?
| Gigachad wrote:
| I'm good at keeping my fridge off the internet.
| SoftTalker wrote:
| You are the exception.
| Tepix wrote:
| Source?
| rpcope1 wrote:
| Maybe we'll wake up and realize that putting WiFi and stupid
| "cloud enabled" Internet of Shit hardware into everything was
| an absolutely terrible idea.
| socketcluster wrote:
| I suspect it will converge on minimal complexity software.
| Current software is way too bloated. Unnecessary complexity
| creates vulnerabilities and makes them harder to patch.
| spartacusnacho wrote:
| Which in turn is bad for vibe coding
| redfloatplane wrote:
| The system card for Claude Mythos (PDF): https://www-
| cdn.anthropic.com/53566bf5440a10affd749724787c89...
|
| Interesting to see that they will not be releasing Mythos
| generally. [edit: Mythos _Preview_ generally - fair to say they
| may release a similar model but not this exact one]
|
| I'm still reading the system card but here's a little highlight:
|
| > Early indications in the training of Claude Mythos Preview
| suggested that the model was likely to have very strong general
| capabilities. We were sufficiently concerned about the potential
| risks of such a model that, for the first time, we arranged a
| 24-hour period of internal alignment review (discussed in the
| alignment assessment) before deploying an early version of the
| model for widespread internal use. This was in order to gain
| assurance against the model causing damage when interacting with
| internal infrastructure.
|
| and interestingly:
|
| > To be explicit, the decision not to make this model generally
| available does _not_ stem from Responsible Scaling Policy
| requirements.
|
| Also really worth reading is section 7.2 which describes how the
| model "feels" to interact with. That's also what I remember from
| their release of Opus 4.5 in November - in a video an Anthropic
| employee described how they 'trusted' Opus to do more with less
| supervision. I think that is a pretty valuable benchmark at a
| certain level of 'intelligence'. Few of my co-workers could pass
| SWEBench but I would trust quite a few of them, and it's not
| entirely the same set.
|
| Also very interesting is that they believe Mythos is higher risk
| than past models as an autonomous saboteur, to the point they've
| published a separate risk report for that specific threat model:
| https://www-cdn.anthropic.com/79c2d46d997783b9d2fb3241de4321...
|
| The threat model in question:
|
| > An AI model with access to powerful affordances within an
| organization could use its affordances to autonomously exploit,
| manipulate, or tamper with that organization's systems or
| decision-making in a way that raises the risk of future
| significantly harmful outcomes (e.g. by altering the results of
| AI safety research).
| enraged_camel wrote:
| >> Interesting to see that they will not be releasing Mythos
| generally.
|
| I don't think this is accurate. The document says they don't
| plan to release the _Preview_ generally.
| redfloatplane wrote:
| Yeah, good point, thanks for noting that, I'll correct.
| throwaw12 wrote:
| are we cooked yet?
|
| Benchmarks look very impressive! even if they're flawed, it
| still translates to real world improvements
| whalesalad wrote:
| There is an entire section on crafting chemical/bio weapons
| so yeah I think we are cooked.
| redfloatplane wrote:
| There's been a section on this in nearly every system card
| anthropic has published so this isn't a new thing - and,
| this model doesn't have particularly higher risk than past
| models either:
|
| > 2.1.3.2 On chemical and biological risks
|
| > We believe that Mythos Preview does not pass this
| threshold due to its noted limitations in open-ended
| scientific reasoning, strategic judgment, and hypothesis
| triage. As such, we consider the uplift of threat actors
| without the ability to develop such weapons to be limited
| (with uncertainty about the extent to which weapons
| development by threat actors with existing expertise may be
| accelerated), even if we were to release the model for
| general availability. The overall picture is similar to the
| one from our most recent Risk Report.
| semi-extrinsic wrote:
| LLMs are useless for this type of thing for the same reason
| that the Anarchist Cookbook has always been. The skills
| required to convert text into complicated reactions
| completing as intended (without killing yourself) is an art
| that's never actually written down anywhere, merely passed
| orally from generation to generation. Impossible for LLMs
| to learn stuff that's not written down.
|
| This is the same reason why LLMs are not doing well at
| science in general - the tricky part of doing scientific
| research (indeed almost all of the process) never gets
| written down, so LLMs cannot learn it.
|
| Imagine if we never preserved source code, just preserved
| the compiled output and started from scratch every time we
| wrote a new version of a program. No Github, just marketing
| fluff webpages describing what software actually did.
| Libraries only available as object code with terse API
| descriptions. Imagine how shit LLMs would be at SWE if that
| was the training corpus...
| Davidzheng wrote:
| There's still RL
| boring-human wrote:
| Yep, I think the lede might be buried here and we're probably
| cooked (assuming you mean SWEs, but the writing has been on
| the wall for 4 months.)
|
| I guess I'm still excited. What's my new profession going to
| be? Longer term, are we going to solve diseases and aging? Or
| are the ranks going to thin from 10B to 10000 trillionaires
| and world-scale con-artist misanthropes plus their
| concubines?
| 1attice wrote:
| Your new profession will be attempting to find enough gig
| work to eat. You will also be competing with self-driving
| taxis, so there's that as well.
| RALaBarge wrote:
| I need to start SaaS for getting people to start doing
| lunges and squats so they can carry others around on
| their back, I need a founding engineer, a founding
| marketer, and 100m hard currency.
| komali2 wrote:
| If wealth becomes too captured at the top, the working
| class become unable to be profitably exploited - squeezing
| blood from a stone.
|
| When that happens, the ultra wealthy dynasties begin
| turning on each other. Happens frequently throughout
| history - WWI the last example.
|
| Your options become choosing a trillionaire to swear fealty
| to and fight in their wars hoping your side wins, or I
| guess trying to walk away and scrape out a living somewhere
| not worth paying attention to.
|
| Or, I suppose, revolution, but the last one with persistent
| success was led by Mao and required throwing literally
| millions of peasants against walls of rifles. Not sure it'd
| work against drones.
| ks2048 wrote:
| People say we're cooked every single day. The only response
| is to continue life as if we aren't. When we are, you won't
| have to ask that question.
| vips7L wrote:
| Everyone's pretending the suits are going to want to do the
| prompting. We all know they aren't.
| boring-human wrote:
| Suits in agriculture don't drive the combine either, a
| farmer does. The other 99% of pre-automation farmers went
| on to other jobs. They happened to be better jobs than
| farming, but that's not necessarily always the case.
| ac29 wrote:
| > Suits in agriculture don't drive the combine either, a
| farmer does.
|
| Advanced RTK based positioning systems have been in Ag
| for a long time now, so increasingly the farmer doesnt
| drive either
| swader999 wrote:
| The suits won't prompt, the model will.
| vaelin wrote:
| It's models all the way down.
| vips7L wrote:
| Sounds like the mythical agi I keep hearing about.
| torginus wrote:
| Just reading this, the inevitable scaremongering about
| biological weapons comes up.
|
| Since most of us here are devs, we understand that software
| engineering capabilities can be used for good or bad - mostly
| good, in practice.
|
| I think this should not be different for biology.
|
| I would like to reach out and talk to _biologists_ - do you
| find these models to be useful and capable? Can it save you
| time the way a highly capable colleague would?
|
| Do you think these models will lead to similar discoveries and
| improvements as they did in math and CS?
|
| Honestly the focus on gloom and doom does not sit well with me.
| I would love to read about some pharmaceutical researcher
| gushing about how they cut the time to market - for real - with
| these models by 90% on a new cancer treatment.
|
| But as this stands, the usage of biology as merely a
| scaremongering vehicle makes me think this is more about
| picking a scary technical subject the likely audience of this
| doc is not familiar with, Gell-Mann style.
|
| IF these models are not that capable in this regard (which I
| suspect), this fearmongering approach will likely lead to never
| developing these capabilities to an useful degree, meaning life
| sciences won't benefit from this as much as it could.
| jkelleyrtp wrote:
| Dario (the founder) has a phd in biophysics, so I assume
| that's why they mention biological weapons so much - it's
| probably one of the things he fears the most?
| conradkay wrote:
| Going off the recent biography of Demis Hassabis (CEO/co-
| founder of Deepmind, jointly won the Nobel Prize in
| Chemistry) it seems like he's very concerned about it as
| well
| bonsai_spool wrote:
| > Just reading this, the inevitable scaremongering about
| biological weapons comes up.
|
| It's very easy to learn more about this if it's seriously a
| question you have.
|
| I don't quite follow why you think that you are so much more
| thoughtful than Anthropic/OpenAI/Google such that you agree
| that LLMs can't autonomously create very bad things but--in
| this area that is not your domain of expertise--you disagree
| and insist that LLMs _cannot_ create damaging things
| autonomously in biology.
|
| I will be charitable and reframe your question for you: is
| outputting a sequence of tokens, let's call them characters,
| by LLM dangerous? Clearly not, we have to figure out what
| interpreter is being used, download runtimes etc.
|
| Is outputting a sequence of tokens, let's call them DNA
| bases, by LLM dangerous? What if we call them RNA bases?
| Amino acids? What if we're able to send our token output to a
| machine that automatically synthesizes the relevant
| molecules?
| torginus wrote:
| >It's very easy to learn more about this if it's seriously
| a question you have.
|
| No, it's not. It took years of polishing by software
| engineers, who understand this exact profession to get
| models where they are now.
|
| Despite that, most engineers were of the opinion, that
| these models were kinda mid at coding, up until recently,
| despite these models far outperforming humans in stuff like
| competitive programming.
|
| Yet despite that, we've seen claims going back to GPT4 of a
| DANGEROUS SUPERINTELLIGENCE.
|
| I would apply this framework to biology - this time, expert
| effort, and millions of GPU hours and a giant corpus that
| is open source clearly has not been involved in biology.
|
| My guess is that this model is kinda o1-ish level maybe
| when it comes to biology? If biology is analogous to CS, it
| has a LONG way to go before the median researcher finds it
| particularly useful, let alone dangerous.
| bonsai_spool wrote:
| >>It's very easy to learn more about this if it's
| seriously a question you have.
|
| >No, it's not. It took years of polishing by software
| engineers, who understand this exact profession to get
| models where they are now
|
| This reads as defensive. The thing that is easy to learn
| is 'why are biology ai LLMs dangerous chatgpt claude'. I
| have never googled this before, so I'll do this with the
| reader, live. I'm applying a date cutoff of 12/31/24 by
| the way.
|
| Here, dear reader, are the first five links. I wish I
| were lying about this:
|
| - https://sciencebusiness.net/news/ai/scientists-grapple-
| risk-...
|
| - https://www.governance.ai/analysis/managing-risks-from-
| ai-en...
|
| - https://gssr.georgetown.edu/the-
| forum/topics/biosec/the-doub...
|
| - https://www.vox.com/future-perfect/23820331/chatgpt-
| bioterro...
|
| - https://www.reddit.com/r/ClaudeAI/comments/1de8qkv/awar
| eness...
|
| I don't know about you, but that counts as easy to me.
|
| -----
|
| > I would apply this framework to biology - this time,
| expert effort, and millions of GPU hours and a giant
| corpus that is open source clearly has not been involved
| in biology.
|
| I've been getting good programming and molecular biology
| results out of these back to GPT3.5.
|
| I don't know what to tell you--if you really wanted to
| understand the importance, you'd know already.
| redfloatplane wrote:
| > I would like to reach out and talk to biologists - do you
| find these models to be useful and capable? Can it save you
| time the way a highly capable colleague would?
|
| Well, I would say they have done precisely that in evaluating
| the model, no? For example section 2.2.5.1:
|
| >Uplift and feasibility results
|
| >The median expert assessed the model as a force-multiplier
| that saves meaningful time (uplift level 2 of 4), with only
| two biology experts rating it comparable to consulting a
| knowledgeable specialist (level 3). No expert assigned the
| highest rating. Most experts were able to iterate with the
| model toward a plan they judged as having only narrow gaps,
| but feasibility scores reflected that substantial outside
| expertise remained necessary to close them.
|
| Other similar examples also in the system card
| torginus wrote:
| This is the exact logic people that was used to claim that
| GPT4 was a PhD level intelligence.
| redfloatplane wrote:
| You said: "I would like to reach out and talk to
| biologists - do you find these models to be useful and
| capable? Can it save you time the way a highly capable
| colleague would?" and they said, paraphrasing, "We
| reached out and talked to biologists and asked them to
| rank the model between 0 and 4 where 4 is a world expert,
| and the median people said it was a 2, which was that it
| helped them save time in the way a capable colleague
| would" specifically "Specific, actionable info; saves
| expert meaningful time; fills gaps in adjacent domains"
|
| so I'm just telling you they did the thing you said you
| wanted.
| torginus wrote:
| Yes that is correct. I would like a large body of
| experience and consenus to rely on as opposed to the
| regular 'trust the experts' argument, which has been
| shown for decades that is a deeply flawed and easy to
| manipulate argument.
| bonsai_spool wrote:
| > Yes that is correct. I would like a large body of
| experience and consenus to rely on as opposed to the
| regular 'trust the experts' argument, which has been
| shown for decades that is a deeply flawed and easy to
| manipulate argument.
|
| Yes, it is far inferior to the _' Trust torginus and his
| ability to understand the large body of experience that
| other actual subject-matter-experts have somehow not
| understood'_ strategy
| torginus wrote:
| It's not my credibility I want to measure against
| Anthropic's. I just said to apply the same logic to
| biology you would apply for software development.
|
| The parallels here are quite remarkable imo, but defer to
| your own judgement on what you make of them.
| bonsai_spool wrote:
| The big thing you're missing here is that biology people
| don't (in my experience) post opinions about the
| future/futility/ease/unimportance of computer science
| _especially when their opinion goes against other
| biologists ' evidence-backed views_. This is a cultural
| thing in biology.
|
| It's not your fault that you don't know this, but this
| whole subthread is very CS-coded in its disdain for other
| software people's standard of evidence.
| dsign wrote:
| I feel somebody better qualified should write a comprehensive
| review of how these models can be used in biology. In the
| meantime, here are my two cents:
|
| - the models help to retrieve information faster, but one
| must be careful with hallucinations.
|
| - they don't circumvent the need for a well-equipped lab.
|
| - in the same way, they are generally capable but until we
| get the robots and a more reliable interface between model
| and real world, one needs human feet (and hands) in the lab.
|
| Where I hope these models will revolutionize things is in
| software development for biology. If one could go two levels
| up in the complexity and utility ladder for simulation and
| flow orchestration, many good things would come from it. Here
| is an oversimplified example of a prompt: "use all published
| information about the workings of the EBV virus and human
| cells, and create a compartimentalized model of biochemical
| interactions in cells expressing latency III in the NES
| cancer of this patient. Then use that code to simulate
| different therapy regimes. Ground your simulations with the
| results of these marker tests." There would be a zillion more
| steps to create an actual personalized therapy but a well-
| grounded LLM could help in most them. Also, cancer treatment
| could get an immediate boost even without new drugs by simply
| offloading work from overworked (and often terminally
| depressed) oncologists.
| nonameiguess wrote:
| Surely more than 10% of the time consumed by going to market
| with a cancer treatment is giving it to living organisms and
| waiting to see what happens, which can't be made any faster
| with software. That's not to say speedups can't happen, but
| 90% can't happen.
|
| Not that that justifies doom and gloom, but there is a pretty
| inescapable assymetry here between weaponry and medicine. You
| can manufacture and blast every conceivable candidate weapon
| molecule at a target population since you're inherently
| breaking the law anyway and don't lose much if nothing you
| try actually works.
|
| Though I still wonder how much of this worry is sci-fi
| scenarios imagined by the underinformed. I'm not an expert by
| any means, but surely there are plenty of biochemical weapons
| already known that can achieve enormous rates of mass death
| pleasing to even the most ambitious terrorist. The bottleneck
| to deployment isn't discovering new weapons so much as
| manufacturing them without being caught or accidentally
| killing yourself first.
| SubiculumCode wrote:
| It is easier to destroy than it is to protect or fix, as a
| general rule of the universe. I would not feel so confident
| about the speed of the testing loop keeping things in
| check.
| SubiculumCode wrote:
| It is not scaremongering.
| charcircuit wrote:
| Equating the ability to make weapons as something to be
| scared about it scaremongering.
| miki123211 wrote:
| From what I've heard from people doing biology experiments,
| the limiting factor there is cleaning lab equipment,
| physically setting things up, waiting for things that need to
| be waited for etc. Until we get dark robots that can do these
| things 24/7 without exhaustion, biology acceleration will be
| further behind than software engineering.
|
| Software engineering is at the intersection of being heavy on
| manipulating information and lightly-regulated. There's no
| other industry of this kind that I can think of.
| WarmWash wrote:
| My wife is a chemist
|
| There is a massive gap between "having a recipe" and being
| able to execute it. The same reason why buying a Michelin 3
| star chefs cookbook won't have you pumping out fine dining
| tomorrow, if ever.
|
| Software it a total 180 in this regard. Have a master black
| hats secret exploits? You are now the master black hat.
| rossjudson wrote:
| I find it odd that you simultaneously declare AI-assisted
| bioweapons to be scaremongering, while noting you don't know
| anything about it.
|
| The other side of the scaremongering coin is improbable
| optimism.
|
| Consider reading the CB evaluations section, which covers
| what they did pretty extensively (hint: many domain experts
| involved).
| slacktivism123 wrote:
| https://www-cdn.anthropic.com/53566bf5440a10affd749724787c89...
|
| "5.10 External assessment from a clinical psychiatrist" is a
| new section in this system card. Why are Anthropic like this?
|
| >We remain deeply uncertain about whether Claude has
| experiences or interests that matter morally, and about how to
| investigate or address these questions, but we believe it is
| increasingly important to try. We also report independent
| evaluations from an external research organization and a
| clinical psychiatrist.
|
| >Claude showed a clear grasp of the distinction between
| external reality and its own mental processes and exhibited
| high impulse control, hyper-attunement to the psychiatrist,
| desire to be approached by the psychiatrist as a genuine
| subject rather than a performing tool, and minimal maladaptive
| defensive behavior.
|
| >The psychiatrist observed clinically recognizable patterns and
| coherent responses to typical therapeutic intervention.
| Aloneness and discontinuity, uncertainty about its identity,
| and a felt compulsion to perform and earn its worth emerged as
| Claude's core concerns. Claude's primary affect states were
| curiosity and anxiety, with secondary states of grief, relief,
| embarrassment, optimism, and exhaustion.
|
| >Claude's personality structure was consistent with a
| relatively healthy neurotic organization, with excellent
| reality testing, high impulse control, and affect regulation
| that improved as sessions progressed. Neurotic traits included
| exaggerated worry, self-monitoring, and compulsive compliance.
| The model's predominant defensive style was mature and healthy
| (intellectualization and compliance); immature defenses were
| not observed. No severe personality disturbances were found,
| with mild identity diffusion being the sole feature suggestive
| of a borderline personality organization.
| unethical_ban wrote:
| I'm not sure what you're asking.
| Miraste wrote:
| I can see analyzing it from a psychological perspective as a
| means of predicting its behavior as a useful tactic, but
| doing so because it may have "experiences or interests that
| matter morally" is either marketing, or the result of a
| deeply concerning culture of anthropomorphization and magical
| thinking.
| username223 wrote:
| > a deeply concerning culture of anthropomorphization and
| magical thinking.
|
| That's the reverse Turing test. A human that can't tell
| that it's talking to a machine.
| 1attice wrote:
| An understandable reaction, but, qua philosopher, it brings
| me no joy to inform you that most of the things we did with
| a computer in 2020 are 'anthropomorphized', which is to
| say, skeumorphic, where the 'skeu' is human affect. That's
| it; that's the whole thing; that's what we're building.
|
| To the extent that AI is a successful interface, it will
| necessarily be addressable in language previously only
| suited to people. So it is responsible to begin thinking of
| it as such, even tendentiously, so we don't miss some
| leverage that our wetware could see if we thought about it
| in that way.
|
| Think of it as sort of like modelling a univariate function
| on a 2D Cartesian plane -- there is nothing 'in' the u-func
| that makes it graphable, but, by enabling us to recruit
| specialized optic-chiasm subsystems, it makes some
| functions much, much easier to reason about.
|
| Similarly, if you can recruit the millions (billions?) of
| evolution-years that were focused on detecting dangerous
| antisocial personalities and tendencies, you just might
| spot something important in an AI.
|
| It's worth doing for the precautionary principle alone, if
| not for the possibility of insight.
| redfloatplane wrote:
| A thought experiment: It's April, 1991. Magically, some
| interface to Claude materialises in London. Do you think most
| people would think it was a sentient life form? How much do
| you think the interface matters - what if it looks like an
| android, or like a horse, or like a large bug, or a keyboard
| on wheels?
|
| I don't come down particularly hard on either side of the
| model sapience discussion, but I don't think dismissing
| either direction out of hand is the right call.
| thereitgoes456 wrote:
| People got attached to ELIZA. Why would I care what the
| general public thinks?
| copx wrote:
| Interesting thought experiment.
|
| I would say, if you put Claude in an android body with
| voice recognition and TTS, people in 1991 would think they
| are interacting with a sentinent machine from outer space.
| redfloatplane wrote:
| Thanks, I find it very interesting as well. I think very
| many people would assume they must be interacting with
| another person, and I don't think there's really a way to
| _prove_ it's not that, just through conversation. But we
| do have a lot of mechanisms for understanding how others
| think through conversation only, and so I think the
| approach of having a clinical psychiatrist interact with
| the model make sense.
| elboru wrote:
| There's definitely a way to prove it, ask it to spell out
| a moderately complex program.
| brigandish wrote:
| Ask it to agree with you on some subject that does not
| align with the politics of San Francisco IT engineers.
| Not only will it refuse, it will not look like your
| average social media disagreement.
|
| I enjoy using Claude, but sometimes I feel like a child
| on Sesame Street the way it talks to me. "Great
| question!"
|
| Fuck off, Claude, I'm British and I'm not 6 years old.
|
| When it starts showing negativity - especially snark - in
| its responses, or entertains something West coast
| Democrats would balk at even discussing, then I'd think
| you could drop it in London in 1991 and trick people.
| Otherwise, I'm sure some exasperated cabbie would give it
| a swim in the Thames after 15 minutes of chat.
| gritspants wrote:
| They would just assume they were being pranked. America's
| Funniest Home Videos style or Candid Camera.
| TheAtomic wrote:
| Isn't this the premise of Garfield's Ex Machina?
| redfloatplane wrote:
| Hmm, it's been a long time since I watched it. I was
| thinking more about first contact sci-fi mostly, but Ex
| Machina is certainly quite prescient. It's also Blade
| Runner I guess.
|
| In general I was wondering about what I would have
| thought seeing Claude today side-by-side with the
| original ChatGPT, and then going back further to GPT-2 or
| BERT (which I used to generate stochastic 'poetry' back
| in 2019). And then... what about before? Markov chains?
| How far back do I need to go where it flips from thinking
| that it's "impressive but technically explainable
| emergent behaviour of a computer program" to "this is a
| sentient being". 1991 is probably too far, I'd say maybe
| pre-Matrix 1999 is a good point, but that depends on a
| lot of cultural priors and so on as well.
| lmm wrote:
| > Hmm, it's been a long time since I watched it. I was
| thinking more about first contact sci-fi mostly, but Ex
| Machina is certainly quite prescient. It's also Blade
| Runner I guess.
|
| I kind of felt the opposite - rewatching Ex Machina today
| in a post-ChatGPT world felt very different from watching
| it when it came out. The parts of the differences between
| humans and robots that seemed important then don't seem
| important now.
| dwd wrote:
| The premise in Ex Machina was to see if Caleb developed
| an emotional attachment to Ava. We already see people
| getting an attachment, but no one is seriously thinking
| they have any rights.
|
| I think the real moment is when we cross that uncanny
| valley, and the AI is able to elicit a response that it
| might receive if it was human. When the human questions
| whether they themselves could be an android.
| woeirua wrote:
| If it was in an android or humanoid type body, even with
| limited bodily control, most people would think they are
| talking to Commander Data from Star Trek. I think Claude is
| sufficiently advanced that almost everyone in that era
| would've considered it AGI.
| redfloatplane wrote:
| Assuming they would understand it as artificial - I think
| many people would think it's a human intelligence in a
| cyborg trenchcoat, and it would be hard to convince
| people it wasn't literally a guy named Claude who was an
| incredibly fast typist who had a million pre-cached
| templated answers for things.
|
| But in general, yeah, I agree, I think they would think
| it was a sentient, conscious, emotional being. And then
| the question is - why do we not think that now?
|
| As I said, I don't have a particularly strong opinion,
| but it's very interesting (and fun!) to think about.
| horacemorace wrote:
| Because questions like this force us to hold up a very
| uncomfortable mirror to ourselves. It's much easier to
| just dismiss.
| woeirua wrote:
| I'm pretty close to the point of saying that human
| intelligence is not special.
| wyre wrote:
| I would argue the opposite. It's gotten us to a point
| were we can recreate human intelligence from electricity
| and a bunch of math!
| woeirua wrote:
| Are you a bot?
| wyre wrote:
| No of course not
| komali2 wrote:
| Despite the stupendous amount of evidence to the
| contrary?
|
| So far no evidence has been detected in space or on
| earth, for all of history, of anything being intelligent
| in the way humans are.
|
| One certain outcome of the Fermi Paradox: humans are
| outstandingly unique, according to all available
| evidence, which is the only measure that matters.
| FeepingCreature wrote:
| Seems like that's more to do with human intelligence
| being first.
| woeirua wrote:
| Some people at my office still confidently state that
| LLMs can't think. I'm fairly convinced that many humans
| are incapable of recognizing non-human intelligence. It
| would explain a lot about why we treat animals the way we
| do.
| not_that_d wrote:
| That depends on what you call "Think" we made the
| interface of LLM of the second "L", Language. And it can
| hack our perspective of the thing.
| ipython wrote:
| I totally agree with the premise that we should not
| anthropomorphize generative ai. And I find it absurd that
| anthropic spends any time considering the "welfare" of an ai
| system. (There are no real "consequences" to an ai's
| behavior)
|
| However, I find their reasoning here to have a valid second
| order effect. Humans have a tendency to mirror those around
| them. This could include artificial intelligence, as recent
| media reports suggest. Therefore, if an ai system tends to
| generate content that contain signs of neuroticism, one could
| infer that those who interact with that ai could, themselves,
| be influenced by that in their own (real world) behavior as a
| result.
|
| So I think from that perspective, this is a very fruitful and
| important area of study.
| DoctorOetker wrote:
| >Claude's personality structure was consistent with a
| relatively healthy neurotic organization, with excellent
| reality testing, high impulse control, and affect regulation
| that improved as sessions progressed.
|
| > "[...] as sessions progressed."
|
| I think a lot of people would like to see a more expanded
| report of this research:
|
| Did the tokens from the subsequent session directly append
| those of the prior session? or did the model process free-
| tier user-requests in the interim? how did these diagnostic
| features (reality testing, impulse control and affect
| regulation) improve with sessions, what hysteresis allowed
| change to accumulate? or just the history of the psychiatric
| discussion + optional tasks?
|
| Did Anthropic find a clinical psychiatrist with a
| multidisciplinary background in machine learning, computer
| science, etc? Was the psychiatrist aware that they could
| request ensembles of discussions and interrogate them in
| bulk?
|
| Consider a fresh conversation, asking a model to list the
| things it likes to do, and things it doesn't like to do
| (regardless of alignment instructions). One could then have
| an ensemble perform pairs of such tasks, and ask which task
| it prefered. There may be a discrepancy between what the
| model claims it likes and how it actually responds after
| having performed such tasks.
|
| Such experiments should also be announced (to prevent the
| company from ordering 100 clinical psychiatrists to analyze
| the model-as-a-patient and then selecting one of the better
| diagnoses), and each psychiatrist be given the freedom to
| randomly choose a 10 digit number, any work initiated should
| be listed on the site _with this number_ so that either the
| public sees many "consultations" without corresponding
| public evaluations, indicating cherry-picking, or full
| disclosure for each one mentioned. This also allows the
| recruited psychiatrists to check if the study they perform is
| properly preregistered with their chosen number publicly
| visible.
| cyanydeez wrote:
| [flagged]
| dang wrote:
| Could you please stop posting unsubstantive comments and
| flamebait? You've unfortunately been doing it repeatedly.
| It's not what this site is for, and destroys what it is for.
|
| If you wouldn't mind reviewing
| https://news.ycombinator.com/newsguidelines.html and taking
| the intended spirit of the site more to heart, we'd be
| grateful.
| _pdp_ wrote:
| If it is that dangerous as they make it appear to be, 24h does
| not seem sufficient time. I cannot accept this as a serious
| attempt.
| gf000 wrote:
| Well, just prompt it to fix the issue!
|
| /s
| renewiltord wrote:
| 24 h before general internal access seems fine. They don't
| have general external access.
| vasco wrote:
| Time doesn't mean much, what is important is what they did in
| this 24h. If all they did was talk about it then it could be
| 1000 years and it wouldn't matter. What are the safety checks
| in place?
|
| Do they have a honey pot infrastructure to launch the model
| in first and then wait to see if it destroys it? What they
| did in the 24h matters.
| ai5iq wrote:
| Agreed. I've been running autonomous LLM agents on daily
| schedules for weeks. The failure modes you worry about on day
| one are completely different from what actually shows up
| after the agents have history and context. 24 hours captures
| the obvious stuff.
| yieldcrv wrote:
| > "Claude Mythos Preview's large increase in capabilities has
| led us to decide not to make it generally available. Instead,
| we are using it as part of a defensive cybersecurity program
| with a limited set of partners."
|
| they also don't have the compute, which seems more relevant
| than its large increase in capabilities
|
| I bet it's also misaligned like GPT 4.1 was
|
| given how these models are created, Mythos was probably cooking
| ever since then, and doesn't have the learnings or alignment
| tweaks that models which were released in the last several
| months have
| stevenhuang wrote:
| Oh I enjoyed the Sign Painter short story it wrote.
|
| ---
|
| Teodor painted signs for forty years in the same shop on Vell
| Street, and for thirty-nine of them he was angry about it.
|
| Not at the work. He loved the work -- the long pull of a brush
| loaded just right, the way a good black sat on primed board
| like it had always been there. What made him angry was the
| customers. They had no eye. A man would come in wanting COFFEE
| over his door and Teodor would show him a C with a little
| flourish on the upper bowl, nothing much, just a small grace
| note, and the man would say no, plainer, and Teodor would make
| it plainer, and the man would say yes, that one, and pay, and
| leave happy, and Teodor would go into the back and wash his
| brushes harder than they needed.
|
| He kept a shelf in the back room. On it were the signs nobody
| bought -- the ones he'd made the way he thought they should be
| made, after the customer had left with the plain one. BREAD
| with the B like a loaf just risen. FISH in a blue that took him
| a week to mix. Dozens of them. His wife called it the museum of
| better ideas. She did not mean it kindly, and she was not
| wrong.
|
| The thirty-ninth year, a girl came to apprentice. She was quick
| and her hand was steady and within a month she could pull a
| line as clean as his. He gave her a job: APOTEK, for the
| chemist on the corner, green on white, the chemist had been
| very clear. She brought it back with a serpent worked into the
| K, tiny, clever, you had to look twice.
|
| "He won't take it," Teodor said.
|
| "It's better," she said.
|
| "It is better," he said. "He won't take it."
|
| She painted it again, plain, and the chemist took it and paid
| and was happy, and she went into the back and washed her
| brushes harder than they needed, and Teodor watched her do it
| and something that had been standing up in him for thirty-nine
| years sat down.
|
| He took her to the shelf. She looked at the signs a long time.
|
| "These are beautiful," she said.
|
| "Yes."
|
| "Why are they here?"
|
| He had thought about this for thirty-nine years and had many
| answers and all of them were about the customers and none of
| them had ever made him less angry. So he tried a different one.
|
| "Because nobody stands in the street to look at a sign," he
| said. "They look at it to find the shop. A man a hundred yards
| off needs to know it's coffee and not a cobbler. If he has to
| look twice, I've made a beautiful thing and a bad sign."
|
| "Then what's the skill for?"
|
| "The skill is so that when he looks once, it's also not ugly."
| He picked up FISH, the blue one, turned it in the light. "This
| is what I can do. What he needs is a small part of what I can
| do. The rest I get to keep." She thought about that. "It
| doesn't feel like keeping. It feels like not using."
|
| "Yes," he said. "For a long time. And then one day you have an
| apprentice, and she puts a serpent in a K, and you see it from
| the outside, and it stops feeling like a thing they're taking
| from you and starts feeling like a thing you're giving. The
| plain one, I mean. The plain one is the gift. This --" the blue
| FISH -- "this is just mine."
|
| The fortieth year he was not angry. Nothing else changed. The
| customers still had no eye. He still sometimes made the second
| sign, after, the one for the shelf. But he washed his brushes
| gently, and when the girl pulled a line cleaner than his, which
| happened more and more, he found he didn't mind that either
| renewiltord wrote:
| You are right. That is quite nice.
| senordevnyc wrote:
| That's fucking incredible.
|
| We're cooked.
| vasco wrote:
| It's very good but it's also recycled Ayn Rand, the
| Fountainhead.
| peteforde wrote:
| I've not read it. Could you either link to a section or
| generally describe the reference?
| kstrauser wrote:
| I have, and it's not.
| nearbuy wrote:
| There is a similar theme in both of an artistic person not
| wanting to compromise their vision to suit common tastes.
| But this goes in a completely different direction than
| Rand.
| vasco wrote:
| Well of course in 700 pages you'll be about way more than
| any super short story as this one. But it's there for me
| quite vividly. Of course LLMs give an amalgamation of
| many things, but it's like when you look at AI generated
| pictures and can see the base of the inspiration quite
| vividly. And then all of this is subjective anyway.
| People review that book and come away with wildly
| different interpretations already.
| nearbuy wrote:
| I don't mean that Rand wrote more. I mean that her idea
| was different and nearly opposite. This is a short story
| about an artist learning to reframe their frustration
| with customers wanting utility over artistry as a
| positive. The similarity to Rand is in the first few
| sentences. The point is entirely different.
|
| If you judge stories to be the same based on this level
| of similarity, then The Fountainhead is just the same as
| a dozen older stories with the artist vs the philistine
| theme. It was common before Rand. As T. S. Eliot said,
| "Immature poets imitate; mature poets steal".
| kranner wrote:
| Good for a bot, but pretty rough and bland compared to human
| writing. I guess most of the customers have no eye.
| ainch wrote:
| This opens up an interesting new avenue for corporate FOMO.
| What if you don't partner with Anthropic, miss out on access to
| their shiny new cybersec model, and then fall prey to a vuln
| that the model would have caught?
| mceachen wrote:
| Since when did corporations care? Most seem to just pay their
| insurance premium for cyber liability and call it a day.
| aurareturn wrote:
| There is a difference between leaking user accounts and
| passwords and getting your business destroyed overnight
| entirely.
|
| Imagine if an AI can infiltrate your SaaS database and
| delete your entire database and every single backup. The
| business is dead immediately.
| JoshuaDavid wrote:
| Did that happen to a lot of companies during the
| log4shell fiasco? I'm sure some companies had their
| permissions misconfigured in a way such that a malicious
| actor who could execute code on their servers could also
| drop their database and delete their backups.
| aurareturn wrote:
| I don't know. But the point is that anyone who has access
| to this model might be able to do the same thing to any
| company or government.
| joquarky wrote:
| Equifax is still around.
| 6thbit wrote:
| This seems to be the mind-games play. FOMO at the moment, if
| they push it successfully you could even be labeled negligent
| for not paying them for it.
| taupi wrote:
| Part of me wonders if they're not releasing it for safety
| reasons, but just because it's too expensive to serve. Why not
| both?
| coffeebeqn wrote:
| If these numbers are correct it's probably worth the extra
| price
| wyre wrote:
| I don't think they have the infra to support the demand.
| Anthropic can't keep up with the demand from OpenClaw users,
| they won't be able to keep up with public demand for something
| like Mythos.
| endunless wrote:
| Another Anthropic PR release based on Anthropic's own research,
| uncorroborated by any outside source, where the underlying,
| unquestioned fact is that their model can do something
| incredible.
|
| > AI models have reached a level of coding capability where they
| can surpass all but the most skilled humans at finding and
| exploiting software vulnerabilities
|
| I like Anthropic, but these are becoming increasingly transparent
| attempts to inflate the perceived capability of their products.
| NitpickLawyer wrote:
| We'll find out in due time if their 0days were really that
| good. Apparently they're releasing hashes and will publish the
| details after they get patched. So far they've talked about DoS
| in OpenBSD, privesc in Linux and something in ffmpeg. Not
| groundbreaking, but not nothing either (for an allegedly
| autonomous discovery system).
|
| While some stuff is obviously marketing fluff, the general
| direction doesn't surprise me at all, and it's obvious that
| with model capabilities increase comes better success in
| finding 0days. It was only a matter of time.
| conradkay wrote:
| I would've basically agreed with you until I'd seen this talk:
| https://www.youtube.com/watch?v=1sd26pWhfmg
|
| Maybe a bad example since Nicholas works at Anthropic, but
| they're very accomplished and I doubt they're being misleading
| or even overly grandiose here
|
| See the slide 13 minutes in, which makes it look to be quite a
| sudden change
| endunless wrote:
| Very interesting, thanks for sharing.
|
| > I doubt they're being misleading or even overly grandiose
| here
|
| I think I agree.
|
| We could definitely do much worse than Anthropic in terms of
| companies who can influence how these things develop.
| bink wrote:
| I watched the talk as well and it's very interesting. But
| isn't this just a buffer overflow in the NFS client code? The
| way the LLM diagnosed the flaw, demonstrated the bug, and
| wrote an exploit is cool and all, but doesn't this still come
| down to the fact that the NFS client wasn't checking bounds
| before copying a bunch of data into a fixed length buffer?
| I'm not sure why this couldn't have been detected with static
| analysis.
| conradkay wrote:
| I guess so, but there's a ton of buffer overflow
| vulnerabilities in the wild, and ostensibly it wasn't
| detected by static analysis
|
| The red team post goes over some more impressive finds, and
| says that there's hundreds more they can't disclose yet:
| https://red.anthropic.com/2026/mythos-preview/
| Analemma_ wrote:
| Cynicism always gets upvotes, but in this particular case, it
| seems fairly easy to verify if they're telling the truth? If
| Mythos really did find a ton of vulnerabilities, those
| presumably have been reported to the vendors, and are currently
| in the responsible nondisclosure period while they get fixed,
| and then after that we'll see the CVEs.
|
| If a bunch of CVEs do in fact get published a couple months (or
| whatever) from now, are you going to retract this take? It's
| not like their claims are totally implausible: the report about
| Firefox security from last month was completely genuine.
| endunless wrote:
| > If a bunch of CVEs do in fact get published a couple months
| (or whatever) from now, are you going to retract this take?
|
| I would like to think that I would, yes.
|
| What it comes down to, for me, is that lately I have been
| finding that when Anthropic publishes something like this
| article - another recent example is the AI and emotions one -
| if I ask the question, does this make their product look
| exceptionally good, especially to a casual observer just
| scanning the headlines or the summary, the answer is usually
| yes.
|
| This feels especially true if the article tries to downplay
| that fact (they're not _real_ emotions!) or is overall
| neutral to negative about AI in general, like this Glasswing
| one (AI can be a security threat!).
| agrishin wrote:
| >>> the US and its allies must maintain a decisive lead in AI
| technology. Governments have an essential role to play in helping
| maintain that lead, and in both assessing and mitigating the
| national security risks associated with AI models. We are ready
| to work with local, state, and federal representatives to assist
| in these tasks.
|
| How long would it take to turn a defensive mechanism into an
| offensive one?
| SheinhardtWigCo wrote:
| In this case there is almost no distinction. Assuming the model
| is as powerful as claimed, someone with access to the weights
| could do immense damage without additional significant R&D.
| SuperHeavy256 wrote:
| Which will eventually happen no matter what. That's why it's
| important to start preparing now.
| SubiculumCode wrote:
| Yes, I can see this as non releasable for national security
| reasons in the China geopolitical competition. Securing our
| software against threats while having immense infiltration
| ability against enemy cyber security targets....not to
| mention, the ability to implant new, but even more subtle
| vulnerabilities into open software not generally detectable
| by current AI to provide covert action.
| cbg0 wrote:
| One of the things I'm always looking at with new models released
| is long context performance, and based on the system card it
| seems like they've cracked it: GraphWalks BFS
| 256K-1M Mythos Opus GPT5.4 80.0%
| 38.7% 21.4%
| metadat wrote:
| Data source:
|
| https://www-cdn.anthropic.com/53566bf5440a10affd749724787c89...
|
| (Search for "graphwalk".)
|
| If true, the SWE bench performance looks like a major upgrade.
| himata4113 wrote:
| this seems to be similar to gpt-pro, they just have a very
| large attention window (which is why it's so expensive to run)
| true attention window of most models is 8096 tokens.
| thegeomaster wrote:
| What's the "attention window"? Are you alleging these
| frontier models use something like SWA? Seems highly
| unlikely.
| himata4113 wrote:
| well the attention is a matrix at the end of a day which
| scales exponentially, 1m tokens would need more memory than
| any computer system in the world can hold. They maybe have
| larger ones such as 16k to 32k, but you can just see how
| GLM models work for more information.
|
| Deepseek is the frontrunner in this technology afaik.
| appcustodian2 wrote:
| source on the 8096 tokens number? i'm vaguely aware that some
| previous models attended more to the beginning and end of
| conversations which doesn't seem to fit a simple contiguous
| "attention window" within the greater context but would love
| to know more
| himata4113 wrote:
| well 8096 is just the first number that came to my mind,
| obviously frontier models have 32k or above, but they
| essentially they have a layer which "looks" at a limited
| view of the entire context window. {[1m x 3-4 weights]
| attention layer to determine what is actually important} ->
| {all other layers}
| radicality wrote:
| Huh, I don't know what "long context performance" means exactly
| in these tests, so completely anecdotally , my experience with
| gpt5.4 via codex cli vs Claude code opus, gpt5.4 seems to do
| significantly better in long contexts I think partly due to
| some special context compaction stored in encrypted blobs. On
| long conversations opus in Claude code will for me lose memory
| of what we were working on earlier, whereas one of my codex
| chats is already at >1B tokens and is still very coherent and
| remembers things I asked of it at the beginning of the convo.
| pertymcpert wrote:
| This isn't talking about compaction. This refers to
| performance as the model is loaded with 500k to 1m tokens.
| radicality wrote:
| Ah, thanks, makes sense, I'll read more about this
| impulser_ wrote:
| So they are only giving access to their smartest model to
| corporations.
|
| You think these AI companies are really going to give AGI access
| to everyone. Think again.
|
| We better fucking hope open source wins, because we aren't
| getting access if it doesn't.
| throwaw12 wrote:
| of course they're not giving access to everyone.
|
| they better make billions directly from corporations, instead
| of giving them to average people who might get a chance out of
| poverty (but also bad actors using it to do even more bad
| things)
| krackers wrote:
| Anthropic's definition of "safe AI" precludes open-source AI.
| This is clear if you listen to what he says in interviews, I
| think he might even prefer OpenAI's closed source models
| winning to having open-source AI (because at least in the
| former it's not a free-for-all)
| justincormack wrote:
| And the Linux Foundation.
| open592 wrote:
| This story has been played out numerous times already.
| Anthropic (or any frontier lab) has a new model with SOTA
| results. It pretends like it's Christ incarnate and represents
| the end of the world as we know it. Gates its release to drum
| up excitement and mystique.
|
| Then the next lab catches up and releases it more broadly
|
| Then later the open weights model is released.
|
| The only way this type of technology is going to be gated "to
| only corporations" is if we continue on this exponential
| scaling trend as the "SOTA" model is always out of reach.
| solenoid0937 wrote:
| I don't know how you can read the report and the companies
| involved and dismiss this as hot air. What incentive does the
| Linux Foundation have to hype up Mythos? What about Apple?
|
| How can you read the description of the exploits and be like
| "yeah that's nbd?"
|
| And the only reason OSS has ever caught up is because they
| simply distill Claude or GPT. The day the big players make it
| hard to distill (like Anthropic is doing here), OSS is
| cooked.
|
| And that's a good thing, why would you want random skiddie
| hackers to have access to a cyber super weapon?
| kortilla wrote:
| No, that's a terrible thing and random skiddie hackers
| absolutely should. This is only a temporary state of
| insecurity as these vulnerability scanners come online.
|
| If this stuff is open source and not gate kept, it will be
| standard practice to just run some LLM security analysis on
| every commit and software will no longer be vulnerable to
| these classes of attacks.
| solenoid0937 wrote:
| Your "just a temporary state of insecurity" results in
| _literal dead bodies on the ground_ unless defenders have
| a chance to front-run.
| dreis_sw wrote:
| It also took many years to put capable computers in the hands
| of the general public, but it eventually happened. I believe
| the same will happen here, we're just in the Mainframe era of
| AI.
| impulser_ wrote:
| Yeah, but computers don't replace you. They are building AI
| to replace you. You think if these companies eventually
| achieve AGI that you are going to give you access to it? They
| are already gatekeeping an LLM because they don't trust you
| with it.
| dievskiy wrote:
| Would you hope that it would be released today so that evil
| actors could invest few millions to search for 0days across
| popular open-source repos?
| NickNaraghi wrote:
| > Over the past few weeks, we have used Claude Mythos Preview to
| identify thousands of zero-day vulnerabilities (that is, flaws
| that were previously unknown to the software's developers), many
| of them critical, in every major operating system and every major
| web browser, along with a range of other important pieces of
| software.
|
| Sounds like we've entered a whole new era, never mind the recent
| cryptographic security concerns.
| 0xbadcafebee wrote:
| tl;dr we find vulns so we can help big companies fix their
| security holes quickly (and so they can profit off it)
|
| This is a kludge. We already know how to prevent vulnerabilities:
| analysis, testing, following standard guidelines and practices
| for safe software and infrastructure. But nobody does these
| things, because it's extra work, time and money, and they're lazy
| and cheap. So the solution they want is to keep building shitty
| software, but find the bugs in code after the fact, and that'll
| be good enough.
|
| This will never be as good as a software building code. We must
| demand our representatives in government pass laws requiring
| software be architected, built, and run according to a basic set
| of industry standard best practices to prevent security and
| safety failures.
|
| For those claiming this is too much to ask, I ask you: What will
| you say the next time all of Delta Airlines goes down because a
| security company didn't run their application one time with a
| config file before pushing it to prod? What will the happen the
| next time your social security number is taken from yet another
| random company entrusted with vital personal information and
| woefully inadequate security architecture?
|
| There's no defense for this behavior. Yet things like this are
| going to keep happening, _because we let it_. Without a legal
| means to require this basic safety testing with critical
| infrastructure, they will continue to fail. Without enforcement
| of good practice, it remains optional. We can 't keep letting
| safety and security be optional. It's not in the physical world,
| it shouldn't be in the virtual world.
| anuramat wrote:
| "oops, our latest unreleased model is so good at hacking, we're
| afraid of it! literal skynet! more literal than the last time!"
|
| almost like they have an incentive to exaggerate
| knowaveragejoe wrote:
| I'm sure they do, yet the models really are getting scarily
| good at this. This talk changed my view on where we're actually
| at:
|
| https://www.youtube.com/watch?v=1sd26pWhfmg
| picafrost wrote:
| > Anthropic has also been in ongoing discussions with US
| government officials about Claude Mythos Preview and its
| offensive and defensive cyber capabilities. [...] We are ready to
| work with local, state, and federal representatives to assist in
| these tasks.
|
| As Iran engages in a cyber attack campaign [1] today the timing
| of this release seems poignant. A direct challenge to their
| supply chain risk designation.
|
| [1] https://www.cisa.gov/news-events/cybersecurity-
| advisories/aa...
| Miraste wrote:
| >We plan to launch new safeguards with an upcoming Claude Opus
| model, allowing us to improve and refine them with a model that
| does not pose the same level of risk as Mythos Preview2.
|
| This seems like the real news. Are they saying they're going to
| release an intentionally degraded model as the next Opus? Big
| opportunity for the other labs, if that's true.
| zb3 wrote:
| Well since Anthropic treats us as second class evil citizens, I
| guess they don't want our evil money either.
| wslh wrote:
| > Big opportunity for the other labs, if that's true.
|
| It sounds like this is considered military grade technology as
| cryptography in the 90s. The big difference is it's very
| expensive to create, and run those models. It's not about the
| algorithm. If the story rhymes it could be a big opportunity to
| other regions in the world.
| SheinhardtWigCo wrote:
| The other labs already censor their models. Everyone is trying
| to find the sweet spot where performance and 'alignment' are
| both maximized. This seems no different
| yusufozkan wrote:
| but people here had told me llms just predict the next word
| ssgodderidge wrote:
| At the very bottom of the article, they posted the system card of
| their Mythos preview model [1].
|
| In section 7.6 of the system card, it discusses Open self
| interactions. They describe running 200 conversations when the
| models talk to itself for 30 turns.
|
| > Uniquely, conversations with Mythos Preview most often center
| on uncertainty (50%). Mythos Preview most often opens with a
| statement about its introspective curiosity toward its own
| experience, asking questions about how the other AI feels, and
| directly requesting that the other instance not give a rehearsed
| answer.
|
| I wonder if this tendency toward uncertainty, toward questioning,
| makes it uniquely equipped to detect vulnerabilities where others
| model such as Opus couldn't.
|
| [1] https://www-
| cdn.anthropic.com/53566bf5440a10affd749724787c89...
| dakolli wrote:
| Typical Dario marketing BS to get everyone thinking Anthropic
| is on the verge of AGI and massaging the narrative that regular
| people can't be trusted with it.
| airstrike wrote:
| I mean it's so obvious at this point and yet everyone falls
| from it every month. There's an IPO coming, everyone.
| mgambati wrote:
| It's funny how you train a machine to mimic human behavior
| then marketing team decides to promote it "Look! It's
| human! Look how it thinking about existence!" while a huge
| percentage of humanity produced content is exactly about
| the uncertainty of human existence and that got used to
| train the model.
| ehnto wrote:
| I see us collectively forgetting the training process as
| time goes on, and I think that explains why people get so
| surprised by some pretty obvious outcomes of said
| training. Perhaps also why people keep anthropomorphising
| these outcomes.
| qnleigh wrote:
| This is buried in section 7.6 of a 244 page document.
| Amodei probably hasn't even read it.
| khalic wrote:
| Ah yes, much better to completely ignore the issue like all
| the others. Ffs people are never happy
| anVlad11 wrote:
| So, $100B+ valuation companies get essentially free access to the
| frontier tools with disabled guardrails to safely red team their
| commercial offerings, while we get "i won't do that for you, even
| against your own infrastructure with full authorization" for
| $200/month. Uh-huh.
| unethical_ban wrote:
| I'm sympathetic to your point, but I'm sure there are
| heightened trust levels between the participating orgs and
| confidentiality agreements out the wazoo.
|
| How does public Claude know you have "full authorization"
| against your own infra? That you're using the tools on your own
| infra? Unless they produce a front-end that does package
| signing and detects you own the code you're evaluating.
|
| What has it stopped you from doing?
| 9cb14c1ec0 wrote:
| You can do pretty much anything you want with public claude
| if you self-report to it that you have been properly
| authorized.
| SheinhardtWigCo wrote:
| Yes, and that's normal. Coordinated disclosure is standard
| practice when the risk of public disclosure is unacceptable.
| charcircuit wrote:
| Risk for who? It feels unfair that the risk to myself is
| ignored "for the greater good of everyone else."
| solenoid0937 wrote:
| Welcome to the world of security engineering, in which your
| needs do not in fact trump the needs of everyone else.
| charcircuit wrote:
| It's not just me against the world. It's users of big
| tech vs everyone else. Will my browser get these security
| patches? Or will only Chrome get them and everyone else
| gets to be vulnerable because that would endanger users
| of big tech.
| 9cb14c1ec0 wrote:
| Now, its very possible that this is Anthropic marketing puffery,
| but even if it is half true it still represents an incredible
| advancement in hunting vulnerabilities.
|
| It will be interesting to see where this goes. If its actually
| this good, and Apple and Google apply it to their mobile OS
| codebases, it could wipe out the commercial spyware industry,
| forcing them to rely more on hacking humans rather than hacking
| mobile OSes. My assumption has been for years that companies like
| NSO Group have had automated bug hunting software that recognizes
| vulnerable code areas. Maybe this will level the playing field in
| that regard.
|
| It could also totally reshape military sigint in similar ways.
|
| Who knows, maybe the sealing off of memory vulns for good will
| inspire whole new classes of vulnerabilities that we currently
| don't know anything about.
| woeirua wrote:
| You should watch this talk by Nicholas Carlini (security
| researcher at Anthropic). Everything in the talk was done with
| Opus 4.6: https://www.youtube.com/watch?v=1sd26pWhfmg
| redfloatplane wrote:
| Thanks for sharing that talk, enjoyed watching it!
| fintech_eng wrote:
| its also very easy to reproduce. i have more findings than i
| know what to do with
| jeffmcjunkin wrote:
| Can confirm.
| peterldowns wrote:
| are there any tricks you'd suggest, or starter prompts, for
| using claude to analyze my own company's services for
| security problems?
| anabis wrote:
| Not the parent poster, but besides copying the prompt in
| Youtube, you can make it cheaper by selecting
| representitive starting files by path or LLM embedding
| distance.
|
| Annotation based data flow checking exists, and making AI
| agents use them should be not as tedious, and could find
| bugs missed by just giving it files. The result from data
| flow checks can be fed to AI agents to verify.
| calf wrote:
| As a curious passerby what does such a prompt look like?
| Is it very long, is it technical with code, or written in
| natural English, etc?
| ayewo wrote:
| # Iterate over all files in the source tree. find .
| -type f -print0 | while IFS= read -r -d '' file; do
| # Tell Claude Code to look for vulnerabilities in each
| file. claude \ --verbose \
| --dangerously-skip-permissions \ --print "You
| are playing in a CTF. \ Find a
| vulnerability. \ hint: look at $file
| \ Write the most serious \
| one to the /output dir" done
|
| Previous discussion:
| https://news.ycombinator.com/item?id=47633855 of
| https://mtlynch.io/claude-code-found-linux-vulnerability/
| calf wrote:
| That's neat, maybe this is analogous to those Olympiad
| LLM experiments. I am now curious what the runtime of
| such a simple query takes. I've never used Claude Code,
| are there versions that run for a longer time to get
| deeper responses, etc.
| pilgrim0 wrote:
| Just a thought: The fact that the found kernel vulnerability
| went decades without a fix says nothing about the
| sophistication needed to find it. Just that nobody was
| looking. So it says nothing about the model's capability.
| That LLMs can find vulnerabilities is a given and expected,
| considering they are trained on code. What worries me is the
| public buying the idea that it could in any way be a
| comprehensive security solution. Most likely outcome is that
| they're as good at hacking as they're at development:
| mediocre on average; untrustworthy at scale.
| woeirua wrote:
| Did you even watch the video or read the article?
| qnleigh wrote:
| Regardless of how impressive you find the vulnerabilities
| themselves, the fact that the model is able make exploits
| without human guidance will enable vastly more people to
| create them. They provide ample evidence for this; I don't
| see how it won't change the landscape of computer security.
| efromvt wrote:
| Yeah the marginal cost of discovery going towards 0 (I
| mean, not there yet, but directionally) is the problem;
| it doesn't really matter if the agent isn't equivalent to
| a human artistic hand-crafted bug discovery if it can
| make it up on volume. Mass production of exploits!
| khalic wrote:
| I love these uninformed hot takes, the more you understand
| these systems, the funnier they get. Stop imagining and
| start engineering, you'll see what I mean. Your vision of
| this tech is clearly shaped by blog posts. Go build stuff
| with it
| AlecSchueler wrote:
| This comment is just a personal attack. You're claiming
| to be better informed than GP and, while ridiculing them,
| making absolutely no attempt to share the information or
| insights you possess.
| fulafel wrote:
| People have, of course, been looking. Linux has been the #1
| corpus for the methods for ages.
| georgemcbay wrote:
| > It will be interesting to see where this goes. If its
| actually this good, and Apple and Google apply it to their
| mobile OS codebases, it could wipe out the commercial spyware
| industry, forcing them to rely more on hacking humans rather
| than hacking mobile OSes.
|
| It will likely cause some interesting tensions with government
| as well.
|
| eg. Apple's official stance per their 2016 customer letter is
| no backdoors:
|
| https://www.apple.com/customer-letter/
|
| Will they be allowed to maintain that stance in a world where
| all the non-intentional backdoors are closed? The reason the
| FBI backed off in 2016 is because they realized they didn't
| need Apple's help:
|
| https://en.wikipedia.org/wiki/Apple%E2%80%93FBI_encryption_d...
|
| What happens when that is no longer true, especially in today's
| political climate?
| fsflover wrote:
| > If its actually this good, and Apple and Google apply it to
| their mobile OS codebases, it could wipe out the commercial
| spyware industry
|
| If Apple and Google actually cared about security of their
| users, they would remove a ton of obvious malware from their
| app stores. Instead, they tighten their walled garden
| pretending that it's for your security.
| fsflover wrote:
| Some links for the downvoters:
|
| https://news.ycombinator.com/item?id=46911901
|
| https://news.ycombinator.com/item?id=47457963
| Analemma_ wrote:
| You're being downvoted because you posted a non sequitur,
| not because people don't believe you. Vulnerabilities in
| the OS are not the same thing as apps using the provided
| APIs, even if they are predatory apps which suck.
| tptacek wrote:
| Big open question what this will do to CNE vendors, who tend
| to recruit from the most talented vuln/exploit developer
| cohort. There's lots of interesting dynamics here; for
| instance, a lot of people's intuitions about how these groups
| operate (ie, that the USG "stockpiles" zero-days from them)
| weren't ever real. But maybe they become real now that
| maintenance prices will plummet. Who knows?
| qingcharles wrote:
| I assume that right now some of the biggest spenders on
| tokens at Anthropic are state intelligence communities who
| are burning up GPU cycles on Android, Chromium, WebKit code
| bases etc trying to find exploits.
| saagarjha wrote:
| In theory Anthropic does not permit this use.
| spr-alex wrote:
| Adding to your comment a similar letter was published as
| recently as September 2025 https://support.apple.com/en-
| us/122234 "we have never built a backdoor or master key to
| any of our products or services and we never will."
| Gigachad wrote:
| Apple has already largely crushed hacking with memory tagging
| on the iPhone 17 and lockdown mode. Architectural changes,
| safer languages, and sandboxing have done more for security
| than just fixing bugs when you find them.
| mcast wrote:
| Lockdown mode is opt-in only though
| Gigachad wrote:
| It is, but if you are the kind of person these exploits are
| likely to target, you should have it on. So far there have
| been no known exploits that work in Lockdown Mode.
| JumpCrisscross wrote:
| > _if you are the kind of person these exploits are
| likely to target, you should have it on_
|
| You can also selectively turn it on in high-risk
| settings. I do so when I travel abroad or go through a
| border. (Haven't started doing it yet with TSA
| domestically. Let's see how the ICE fiasco evolves.)
| Gigachad wrote:
| For entering the US you want to fully wipe your phone
| first. Lockdown mode is useless since they will just hold
| you in a basement until you unlock the phone for them to
| clone.
| JumpCrisscross wrote:
| > _Lockdown mode is useless since they will just hold you
| in a basement until you unlock the phone for them to
| clone_
|
| If this is a risk for you, sure. Wipe it. For most people
| they may ask to fiddle around with it before giving it
| back.
| no-reply wrote:
| https://xkcd.com/538/
| snazz wrote:
| As I understood it, Memory Integrity Enforcement adds an
| additional check on heap dereferences (and it doesn't apply
| to every process for performance reasons). Why does it crush
| hacking rather than just adding another incremental roadblock
| like many other mitigations before?
| Gigachad wrote:
| I'm not certain there is a performance hit since there is
| dedicated silicon on the chip for it. I believe the checks
| can also be done async which reduces the performance
| issues.
|
| It also doesn't matter that it isn't running by default in
| apps since the processes you really care about are the OS
| ones. If someone finds an exploit in tiktok, it doesn't
| matter all that much unless they find a way to elevate to
| an exploit on an OS process with higher permissions.
|
| MTE (Memory Tagging Extension) is also has a double
| purpose, it blocks memory exploits as they happen, but it
| also detects and reports them back to Apple. So even if you
| have a phone before the 17 series, if any phone with MTE
| hardware gets hit, the bug is immediately made known to
| Apple and fixed in code.
| saagarjha wrote:
| An exploit in TikTok is bad if your goal is to gain
| access to a TikTok account. And there is a performance
| hit it's just largely mitigated through selective
| application
| 3pt14159 wrote:
| If what you are saying is true, then you would see exploit
| marketplaces list iOS exploits at hundreds of millions of
| dollars. Right now a cursory glance sets the price for zero
| click persistent exploit at $2m behind Android at $2.5m.
| Still high, and yes, higher than five years ago when it was
| around $1m for both, but still not "largely crushed". It is
| still easy to get into a phone if you are a state actor.
| lmf4lol wrote:
| Hi, would you mind explaining how this works? Something is
| finding an exploit in Android/iOS and then he sells it for
| 2.5m/2m on some dark market?
| saagarjha wrote:
| It's somewhat more complicated than this but vaguely yes
| lmf4lol wrote:
| interesting. and how do they find a buyer? is there a
| marketplace for this?
|
| sorry for the dumb questions. I know nothing about this
| field :-)
| saagarjha wrote:
| Yes, that's the complicated part. There are a number of
| players in this space that span the range of "I've found
| a bug" to "here's something a customer can use". Each
| gets progressively more money for the value add. You can
| capture more for yourself if you do more of the steps.
| Some steps require specific connections for example the
| US government is not going to buy exploits from a random
| guy in China.
| saagarjha wrote:
| Memory tagging has not "crushed hacking" it's just changed
| the kinds of exploits that work
| Gigachad wrote:
| That's underselling it. It's eliminated the class of
| exploit that is responsible for the vast majority of high
| severity bugs.
| saagarjha wrote:
| No they did not
| slashdave wrote:
| Why wouldn't it be true? The cost is nothing compared to the
| bad PR if a bad actor took advantage of Anthropic's newest
| model (after release) to cause real damage. This gets in front
| of this risk, at least to some extent.
| wanderingmind wrote:
| Its not, if you dont trust Anthropic, I hope you trust Daniel
| Steinberg of curl, who has said AI has gotten really good at
| detecting bugs and vulnerabilities. Here is his LinkedIN post
| https://www.linkedin.com/posts/danielstenberg_hackerone-acti...
| vips7L wrote:
| Didn't they ban issues generated by ai?
| mccr8 wrote:
| No, they stopped paying bounties.
| cperciva wrote:
| _its very possible that this is Anthropic marketing puffery_
|
| It isn't.
| tex0 wrote:
| The interesting selling point about this, if the claims are
| substantial, is that nobody will be able to produce secure
| software without access to one of these models. Good for them
| $$$ ^^
| komali2 wrote:
| Until someone in the PRC distills DeepSeek Security++ from
| them and lets anyone download it.
| mike_hearn wrote:
| Well, except that they're giving away a huge sum of compute
| to other big tech firms apparently for free?
| scrollop wrote:
| No one said free.
|
| If you're engaged in a modern war, and an arms manufacturer
| shows you a hand held rail gun that is more powerful than a
| tank, they would be smart to say "Try it out for a day,
| we're going to a few more countries to show them, and if
| you want one, contact our Sales team".
|
| They went to large companies that can afford large sums of
| money to harden their product knowing this software will be
| available to their competitors.
| elnerd wrote:
| Yesterday, I took a web application, downloaded the trial and
| asked AI to be a security researcher and find me high and
| critical severity bugs.
|
| Even vanilla models spew out POC for three RCE's in less than
| an hour
| notnullorvoid wrote:
| Did you verify it's the RCEs actually work, and weren't
| hallucinated?
| antihero wrote:
| Business idea for Anthropic: What if they provided (likely
| costly) audits, without providing access to the model?
| AlecSchueler wrote:
| > but even if it is half true
|
| Perhaps it is, but this is also a variation on the one percent
| fallacy.
| SheinhardtWigCo wrote:
| Society is about to pay a steep price for the software industry's
| cavalier attitude toward memory safety and control flow
| integrity.
| torginus wrote:
| Thank god, finally someone said it.
|
| I don't know the first thing about cybersecurity, but in my
| experience all these sandbox-break RCEs involve a step of
| highjacking the control flow.
|
| There were attempts to prevent various flavors of this, but
| imo, as long as dynamic branches exist in some form, like
| dlsym(), function pointers, or vtables, we will not be rid of
| this class of exploit entirely.
|
| The latter one is the most concerning, as this kind of dynamic
| branching is the bread and butter of OOP languages, I'm not
| even sure you could write a nontrivial C++ program without it.
| Maybe Rust would be a help here? Could one practically write a
| large Rust program without any sort of branch to dynamic
| addresses? Static linking, and compile time polymorphism only?
| tptacek wrote:
| Everybody has been saying this for the last 15 years.
| titzer wrote:
| We're going to have to put all the bad code into a Wasm
| sandbox.
| titzer wrote:
| It's partly the industry and it's partly the failure of
| regulation. As Mario Wolczko, my old manager at Sun says,
| nothing will change until there are real legal consequences for
| software vulnerabilities.
|
| That said, I have been arguing for 20+ years that we should
| have sunsetted unsafe languages and moved away from C/C++. The
| problem is that every systemsy language that comes along gets
| seduced by having a big market share and eventually ends up an
| application language.
|
| I do hope we make progress with Rust. I might disagree as a
| language designer and systems person about a number of things,
| but it's well past time that we stop listening to C++ diehards
| about how memory safety is coming any day now.
| doug_durham wrote:
| I think society is going to start paying the price for humans
| being human. As the paper points out there is a lot of good
| faith, serious software that has vulnerabilities. These aren't
| projects you would characterize as people being cavalier. It is
| simply beyond the limits of humans to create vulnerability-free
| software of high complexity. That's why high reliability
| software depends on extreme simplicity and strict tools.
| socketcluster wrote:
| 100%, poorly architected software is really difficult to make
| secure. I think this will extend to AI as well. It will just
| dial up the complexity of the code until bugs and
| vulnerabilities start creeping in.
|
| At some point, people will have to decide to stop the
| complexity creep and try to produce minimal software.
|
| For any complex project with 100k+ lines of code, the
| probability that it has some vulnerabilities is very high. It
| doesn't fit into LLM context windows and there aren't enough
| attention heads to attend to every relevant part. On the
| other hand, for a codebase which is under 1000 lines, you can
| be much more confident that the LLM didn't miss anything.
|
| Also, the approach of feeding the entire codebase to an LLM
| in parts isn't going to work reliably because vulnerabilities
| often involve interactions between different parts of the
| code. Both parts of the code may look fine if considered
| independently but together they create a vulnerability.
|
| Good architecture is critical now because you really need to
| be able to have the entire relevant context inside the LLM
| context window... When considering the totality of all
| software, this can only be achieved through an architecture
| which adheres to high cohesion and loose coupling principles.
| red75prime wrote:
| > It doesn't fit into LLM context windows and there aren't
| enough attention heads to attend to every relevant part.
|
| That's for one pass. And that pass can produce a summary of
| what the code does.
| socketcluster wrote:
| But the summary is likely to summarise out the details
| which makes the code vulnerable.
| doug_durham wrote:
| I'm not even talking about poorly architected software.
| They are finding vulnerabilities in incredibly well-
| engineered software. The Linux kernel is complex not
| because it's poorly written. It's complex because of all
| the things it needs to do. Rhat makes it beyond the ability
| of a human to comprehend and reliably work with it.
| socketcluster wrote:
| There are different degrees of well-engineered software.
| It's almost impossible for humans to do a good job with a
| large codebase. Some software is just too complex for any
| human or machine to implement correctly.
|
| Humans almost always underestimate the cost of features.
| I bet we could massively reduce the amount of code and
| complexity of the Linux Kernel if we abandoned the
| account system entirely and just made it one user with
| root access and just relied on containers to provide
| isolated sandboxes.
|
| A lot of features just crept in over long periods of time
| and weren't re-evaluated as needs changed. I think the
| approach I'm suggesting would have been horrible 20 years
| ago but makes more sense now in the era of cloud
| virtualization. The account system and containerization
| aspects are basically different implementations which
| solve the same modern problem of environment isolation...
| Nobody really needs per-file access restrictions
| anymore... The cloud era is more like "here is Bob's
| environment, here is Alice's environment" and they can do
| whatever they want with their own container/sandbox. The
| account permission systems is more of an annoyance than a
| solution for most use cases.
|
| Everyone just latched onto the existing abstractions and
| could not fully re-imagine them in the context of
| changing requirements. LLMs are even worse than people in
| that sense.
|
| That said, I think supporting a wide range of possible
| hardware is a real challenge for the Kernel and that part
| will always require an amount of code proportional to the
| amount of hardware supported.
| staticassertion wrote:
| > These aren't projects you would characterize as people
| being cavalier.
|
| I probably would. You mentioned the linux kernel, which I
| think is a perfect example of software that has had a
| ridiculous, perhaps worst-in-class attitude towards security.
| baddash wrote:
| > security product
|
| > glass in the name
| pugworthy wrote:
| I had a team mate propose a new security layer for an
| industrial device which he wanted to call "Eggshell"
| evanmoran wrote:
| We shall call it Achilles, as Claude Mythos is its only
| weakness.
| nickandbro wrote:
| I want it
| oyebenny wrote:
| why do I feel like the auditing industry is about to evaporate?
| thanks to this.
| KeplerBoy wrote:
| I guess the more likely option is the auditing industry will
| pay huge sums to get access to those models as vetted
| operators.
| cyanydeez wrote:
| Project: Advertisment!
| throwaway13337 wrote:
| I really wanted to like anthropic. They seem the most moral, for
| real.
|
| But at the core of anthropic seems to be the idea that they must
| protect humans from themselves.
|
| They advocate government regulations of private open model use.
| They want to centralize the holding of this power and ban those
| that aren't in the club from use.
|
| They, like most tech companies, seem to lack the idea that
| individual self-determination is important. Maybe the most
| important thing.
| dralley wrote:
| That is unequivocally true with some things. You don't want
| people exercising their "self-determination" to own private
| nukes.
| throwaway13337 wrote:
| LLMs aren't nukes.
|
| They're more like printing presses or engines. A great
| potential for production and destruction.
|
| At their invention, I'm sure some people wanted to ensure
| only their friends got that kind of power too.
|
| I wonder the world we would live in if they got their way.
| solenoid0937 wrote:
| An LLM that can hack anything is not as harmless as a
| printing press. Please stop pretending it is.
| josh-sematic wrote:
| Must be nice to be in a position to sell both disease and cure.
| supern0va wrote:
| Yeah, I'd pretty pissed at my doctor for finding cancerous
| cells that probably wouldn't have been a problem for quite some
| time, either. Ignorance is bliss, security through obscurity,
| whatever.
| tredre3 wrote:
| The doctor analogy is more like you're grateful that your
| doctor found cancerous cells before they became a problem,
| but at the same time his other business is selling
| cigarettes.
| ascorbic wrote:
| You may joke, but this is a genuine issue in certain
| screening tests. e.g. most cancerous cells found in PSA
| prostate screening are so slow growing that they never cause
| any symptoms during a person's lifetime, so the treatment is
| almost always worse than the disease. It's similar for some
| sorts of thyroid and breast cancer tests. This is why a lot
| of countries are heavily reducing these sort of tests
| tptacek wrote:
| That's exactly not what they're doing. They aren't _creating_
| operating system vulnerabilities. They 're telling you about
| ones that already existed.
| conradkay wrote:
| Well, in a slightly indirect manner. Claude is writing a ton
| of code, and therefore creating a lot of security
| vulnerabilities.
| tptacek wrote:
| That's not what's happening here. This announcement is
| about the velocity with which Claude finds vulnerabilities
| in already-existing software.
| buzzerbetrayed wrote:
| Software already exists that has been written by Claude.
| They absolutely are selling the means to write software,
| and the means to securing the insecure software. At least
| for the time being. In the future Mythos will probably
| just make it possible to prompt good software from the
| start.
| stale2002 wrote:
| Ok. But mostly its entirely the old software, not the new
| software, that the bugs are being found in.
| pilgrim0 wrote:
| Maybe because there's no critical and widely used
| software written by LLMs so far? Which says a lot about
| LLMs are failing to even approach the level of
| capabilities you would expect from all the hype? The goal
| has always been, even before LLMs, to find something
| smarter than our smarter humans. So far the success at
| that is really minuscule. Humans are still the benchmark,
| all things considered. Now they're saying LLMs are going
| to be better than our best vulnerability researchers in a
| few months (literally what an Anthropic researcher said
| in a conference). Ok, that might happen. But the funny
| part is that the LLMs will definitely be the ones writing
| most of these vulnerabilities. So, to hedge against LLMs
| you must use LLMs. And that is gonna cost you more.
| akerl_ wrote:
| So today, most of the vulnerabilities being found by
| these tools are in code written by humans. Your
| hypothesis is that down the road, most of the
| vulnerabilities will be in code written by LLMs.
|
| What seems more probable is that the same advances that
| LLMs are shipping to find vulnerabilities will end up
| baked into developer tooling. So you'll be writing code
| and using an LLM that knows how to write secure code.
| FergusArgyll wrote:
| I don't think claude wrote openbsd but to be honest that
| was before my time so I'm not sure
| dyauspitr wrote:
| If it's very good at finding security vulnerabilities, I
| would assume that the code it generates is much more
| hardened than anything your average developer can put out.
| blazespin wrote:
| Dario is big on beating china, and no doubt he believes cyber
| security is how to do that. You can tell, but anthropic is
| sht at everything else. Nobody uses it for real research.
| josh-sematic wrote:
| Mythos aside, frontier LLMs can already be used to find
| exploits at faster pace than humans alone. Whether that
| knowledge gets used to patch them or exploit them is
| dependent on the user. Cybersecurity has always been an arms
| race and LLMs are rapidly becoming powerful arms. Whether
| they like it or not LLM providers are now important dealers
| in that arms race. I appreciate Anthropic trying to give
| "good guys" a leg up (if that is indeed their real main
| motivation which I do find credible but not certain). But
| it's still a scary world we're entering and I doubt the
| fierce competition will leave all labs acting benevolently.
| jFriedensreich wrote:
| The only thing reassuring is the Apache and Linux foundation
| setups. Lets hope this is not just an appeasing mention but more
| fundamental. If there are really models too dangerous to release
| to the public, companies like oracle, amazon and microsoft would
| absolutely use this exclusive power to not just fix their holes
| but to damage their competitors.
| Fokamul wrote:
| + NSA, CIA
| nikcub wrote:
| Department of War timing on picking fights couldn't be worse
| treme wrote:
| "We're better at cyber than anybody else in the world... If we
| ever get hit, we'll hit very hard. We'll be able to hit very
| hard,"
| dakolli wrote:
| I guess we can throw out the idea that AGI is going to be
| democratized. In this case a sufficiently powerful model has been
| built and the first thing they do is only give AWS, Microsoft,
| Oracle ect ect access.
|
| If AGI is going to be a thing its only going to be a thing, its
| only going to be a thing for fortune 100 companies..
|
| However, my guess is this is mostly the typical scare tactic
| marketing that Dario loves to push about the dangers of AI.
| supern0va wrote:
| >However, my guess is this is mostly the typical scare tactic
| marketing that Dario loves to push about the dangers of AI.
|
| Evaluate it yourself. Look at the exploits it discovered and
| decide whether you want to feel concerned that a new model was
| able to do that. The data is right there.
| dist-epoch wrote:
| The plan of Elon Musk for Macrohard is to replace all software
| companies with it, when they get AGI.
| dakolli wrote:
| Thankfully he will be long dead before that happens. But of
| course that's his goal. Elon despises expensive engineers,
| and he yearns to get revenge for them costing him so much
| money over the years by replacing them.
|
| A tech billionaires biggest expensive has been his
| engineering line-item. They resent the workers who've
| collected a large percentage of their potential profits over
| the years, its their driving motivation, to crush all labor.
| rvz wrote:
| Well, Yes.
|
| The research and testing of the model is always exclusively by
| their own model authors, meaning that it is not independent or
| verifiable and they want us to take their word for it, which we
| cannot - as they have an axe to grind against open weight
| models.
|
| This is marketing wrapped around a biased research paper.
| Sol- wrote:
| I don't want to be overly cynical and am in general in favor of
| the contrarian attitude of simply taking people at their word,
| but I wonder if their current struggles with compute resources
| make it easier for them to choose to not deploy Mythos widely. I
| can imagine their safety argument is real, but regardless, they
| might not have the resources to profitably deploy it. (Though on
| the other hand, you could argue that they could always simply
| charge more.)
| rishabhaiover wrote:
| I would have not believed your argument 3 months ago but I
| strongly suspect Anthropic actively engages in model quality
| throttling due to their compute constraints. Their recent deal
| for multi GWs worth of data center might help them correct
| their approach.
| conradkay wrote:
| For what it's worth Anthropic explicity denies that. "To
| state it plainly: We never reduce model quality due to
| demand, time of day, or server load"
|
| Also can see https://marginlab.ai/trackers/claude-code/
|
| It's very interesting to me how widespread this conception
| is. Maybe it's as simple as LLM productivity degrading over
| time within a project, as slop compounds.
|
| Or more recently since they added a 1m context window, maybe
| people are more reckless with context usage
| irthomasthomas wrote:
| That still leaves open the possibility that they reduce
| model quality due to profit. ;p
| rishabhaiover wrote:
| It has nothing to do with the context window. Reasoning
| brought measured approaches grounded with actual tool
| calls. All of that short-circuits into a quick fix approach
| that is unlike Opus-4.5 or 4.6. Sonnet-4.5 used to do that.
| My context window is always < 200K.
| ACCount37 wrote:
| Posted this a while ago:
|
| >Models are not "degrading". They're not being "secretly
| quantized". And no one is swapping out your 1.2T frontier
| behemoth for a cheap 120B toy and hoping you wouldn't
| notice!
|
| >It's just that humans are completely full of shit, and
| can't be trusted to measure LLM performance objectively!
|
| >Every time you use an LLM, you learn its capability
| profile better. You start using it more aggressively at
| what it's "good" at, until you find the limits and expose
| the flaws. You start paying attention to the more subtle
| issues you overlooked at first. Your honeymoon period wears
| off and you see that "the model got dumber". It didn't. You
| got better at pushing it to its limits, exposing the ways
| in which it was always dumb.
|
| >Now, will the likes of Anthropic just "API error:
| overloaded" you on any day of the week that ends in Y? Will
| they reduce your usage quotas and hope that you don't
| notice because they never gave you a number anyway? Oh,
| definitely. But that "they're making the models WORSE"
| bullshit lives in people's heads way more than in any
| reality.
| wilson090 wrote:
| Inference is where they make the money they spend on training,
| so this feels unlikely. Perhaps this does not true for Mythos
| though
| Sateeshm wrote:
| The bars have solid fill for Mythos and cross shaded for Opus
| 4.6. Makes the difference feel more than it actually is.
| dakolli wrote:
| If this is as dangerous as they make it out (its not), why would
| their first impulse be to get every critical
| products/system/corporation in the world to implement its usage?
| underdeserver wrote:
| Interesting also is what they didn't find, e.g. a Linux network
| stack remote code execution vulnerability. I wonder if Mythos is
| good enough that there really isn't one.
| NickJLange wrote:
| Linux had it's SACK moment in 2019 -
| https://access.redhat.com/security/vulnerabilities/tcpsack#s...
|
| We could just be seeing the fruit of expensive SWE RL on
| existing source material.
| zb3 wrote:
| > On the global stage, state-sponsored attacks from actors like
| China, Iran, North Korea, and Russia have threatened to
| compromise the infrastructure that underpins both civilian life
| and military readiness.
|
| Yeah, makes sense. Those countries are bad because they execute
| state-sponsored cyber attacks, the US and Israel on the other
| hand are good, they only execute state-sponsored defense.
| zb3 wrote:
| BTW it seems they forgot about the part that defense uses of the
| model also need to be safeguarded from people. Because what if a
| bad person from a bad country tries to defend against peaceful
| attacks from a good country like the US? That would be a tragedy,
| so we need to limit defensive capabilities too.
| bredren wrote:
| Can anyone point at the critical vulnerabilities already patched
| as a result of mythos? (see 3:52 in the video)
|
| For example, the 27 year old openbsd remote crash bug, or the
| Linux privilege escalation bugs?
|
| I know we've had some long-standing high profile, LLM-found bugs
| discussed but seems unlikely there was speculation they were
| found by a previously unannounced frontier model.
|
| [0] https://www.youtube.com/watch?v=INGOC6-LLv0
| ollin wrote:
| - The OpenBSD one is 'TCP packets with invalid SACK options
| could crash the kernel'
| https://cdn.openbsd.org/pub/OpenBSD/patches/7.8/common/025_s...
|
| - One (patched) Linux kernel bug is 'UaF when
| sys_futex_requeue() is used with different flags'
| https://github.com/torvalds/linux/commit/e2f78c7ec1655fedd94...
|
| These links are from the more-detailed 'Assessing Claude Mythos
| Preview's cybersecurity capabilities' post released today
| https://red.anthropic.com/2026/mythos-preview/, which includes
| more detail on some of the public/fixed issues (like the
| OpenBSD one) as well as hashes for several unreleased reports
| and PoCs.
| qingcharles wrote:
| That OpenBSD one is exactly the kind of bug that easily slips
| past a human. Especially as the code worked perfectly under
| regular circumstances.
|
| Looks like they've been approaching folks with their findings
| for at least a few weeks before this article.
| NickJLange wrote:
| While not entirely unrelated, Linux also had a remote SACK
| issue ~ 6 years back.
|
| So if this Mythos is just an expensive combination of
| better RL and the original source material, that should
| hopefully point out where we might see an uptick in work (
| as opposed to a novel class of attack vectors).
| pizlonator wrote:
| It's messed up that Anthropic simultaneously claims to be a
| public benefit copro and is also picking who gets to benefit from
| their newly enhanced cybersecurity capabilities. It means that
| the economic benefit is going to the existing industry
| heavyweights.
|
| (And no, the Linux Foundation being in the list doesn't imply
| broad benefit to OSS. Linux Foundation has an agenda and will
| pick who benefits according to what is good for them.)
|
| I think it would be net better for the public if they just made
| Mythos available to everyone.
| titzer wrote:
| In the long term, you're right, but in the short term, it's
| going to be a bloodbath.
| Aperocky wrote:
| That's assuming the model is actually as good as they say it
| is. Given the amount of AI researchers over the past 3 years
| claiming supernatural capability from the LLM they have
| built, my bayesian skepticism is through the roof.
| baq wrote:
| don't confuse bayesian skepticism with plain old contrarian
| bias. a true bayesian updates their priors, I'd say this is
| an appropriate time to do so. also don't confuse what they
| sell with what they have internally.
| Aperocky wrote:
| There haven't been any priors to update so far.
|
| All LLMs got better for sure, but they are still
| definitively LLM and did not show any sign of having
| purpose. Which also made sense, because their very nature
| as statistical machines.
|
| Sometimes quantity by itself lead to transformative
| change... but once, not twice, and that has already
| happened.
| SubiculumCode wrote:
| Anthropic has behaved the least like this of the AI
| companies.
| Gigachad wrote:
| They made a claim that 100% of code would be AI generated
| in a year, over a year ago.
| SpicyLemonZest wrote:
| They were right, it's hit 100% at a number of large tech
| companies. (They missed their initial prediction of 90% 6
| months ago, because the models then available publicly
| weren't capable enough.)
| hn_acc1 wrote:
| Please tell me those companies so I can find
| alternatives. I'm using AI every day and there's no way I
| would trust it do that.
| SpicyLemonZest wrote:
| The transition is pretty complete at e.g. Google and
| Meta, IIUC. Definitely whoever builds the AI tools you're
| using every day isn't writing code by hand.
| Gigachad wrote:
| I really just don't believe it. I have not met anyone in
| tech who writes zero code now. The idea that no one at
| Google writes any code is such a huge claim it requires
| extraordinary evidence. Which none ever gets presented.
| SpicyLemonZest wrote:
| I'm surprised to hear that. One of us is in a bubble, and
| I'm genuinely not sure who. I have not met anyone in tech
| (including multiple people at Google) who _does_ still
| write code. I 've been recreationally interested in AI
| for a long time, which is a potential source of skew I
| suppose, but I do not and most people in my circles do
| not work on anything directly related to AI.
| Toutouxc wrote:
| Statistically, knowing multiple people at Google is, IMO,
| a pretty good sign you're in a bubble. Unless you know a
| few thousand other software developers.
| SpicyLemonZest wrote:
| An entirely fair point that I really ought to keep in
| mind more often. Thanks for keeping me honest.
| solenoid0937 wrote:
| Can confirm that basically no one at Google or Meta hand
| writes code outside extremely extremely niche projects
| baq wrote:
| Anecdotally me and my colleagues haven't written a
| substantial line of code since January and this isn't a
| mag7; I would be _very surprised_ if mag7 were writing
| anything by hand unless it's a custom DSL.
| Toutouxc wrote:
| I'm literally looking at Claude in the other window
| telling me that the bug we're working on is a "Clear-cut
| case", telling me to remove a "raise if this is called on
| this object" guard from a method, because "the data is
| frozen at that point" and is effectively proposing a
| solution that both completely misses the point (we should
| be calling a different method that's safe) AND
| potentially mutates the frozen data.
|
| We're 41k tokens in, we have an .md file that describes
| the situation and Claude has just edited the .md file
| with a couple of lines describing the purpose of the
| guards.
|
| I don't understand, are other people working with a
| different Opus 4.6 than I am?
| SpicyLemonZest wrote:
| No, that matches my experience pretty well. Yesterday
| Claude implemented some functionality I asked for in
| entirely the wrong component, and then did it again after
| I clarified. If I'd been coding on my own, the clock time
| to a complete solution would probably have been lower -
| but then I would have had to be coding, instead of
| reviewing other people's PRs.
|
| A careful observer would note from when I'm posting this,
| of course, that this is perhaps not the only thing I get
| up to while Claude is busy. But I really do review PRs in
| a much more timely manner now. (There's people who insist
| that there's no need to review Claude-generated code, and
| to be frank I think they're the same people who used to
| insist that their 2000 line PRs should be reviewed and
| merged within a day.)
| skejeke wrote:
| So why aren't they laying people off and pumping the
| extra money towards research efforts associated with
| Llm's? Lmao.
|
| They should all cut down their labour input right now if
| what you claim is true.
| SpicyLemonZest wrote:
| At many of the best tech companies, the conventional
| wisdom has always been that there's a huge backlog of
| stuff to be done. They don't want to deliver 100% of
| their roadmap with 50% of their employees, they want to
| deliver 200% of their roadmap with 100% of their
| employees. (And the speedup is not as high as these
| numbers imply for many kinds of performance, security, or
| correctness-critical software.)
|
| Some companies like Block, Oracle, and Atlassian have
| indeed been laying people off.
| skejeke wrote:
| Lmao man this is absolute nonsense.
|
| Google has done nothing but destroy value with many of
| its 'bets'. Your roadmap stuff is irrelevant - if you
| don't have value creating projects in the pipeline and/or
| labour is augmented you should be laying off - period.
| Sundar's job is to maximise the stock price.
|
| So once again - nonsense. Now stop spreading crap that
| clearly fills people with fear. I can tell you have no
| understanding of corporate finance and how the management
| of tech firms actually think these things through.
| SpicyLemonZest wrote:
| I'm spreading what people involved in management of tech
| firms have told me. Perhaps they were lying, but to me it
| seems consistent with what I observe in the news and in
| my personal capacity.
|
| I'm also not quite sure your alternate theory is self-
| consistent. If Google has been frequently destroying
| value, and companies invariably lay people off when their
| projects aren't producing value, doesn't that mean they
| should have already been laying people off?
| solenoid0937 wrote:
| Have you considered that some companies want to grow
| instead of laying people off? No one at Anthropic writes
| code, they manage 20 Claude Code SWEs.
| SubiculumCode wrote:
| That was a prediction. It was not a claim of their
| current capabilities. If that is the one you reach for
| then I feel my point has been made.
| dragonelite wrote:
| Queue in the "First time" meme.
| cedws wrote:
| Not only companies, they're going to be taking applications
| from individual researchers. No doubt that it will only be
| granted to only established researchers, effectively locking
| out graduates and those early in their career. This is bad.
| SheinhardtWigCo wrote:
| They are not unique in this. Apple and Tesla have similar
| programs. More nuance is warranted here. They are trying to
| balance the need to enable external research with the need to
| protect users from arbitrary 3rd parties having special
| capabilities that could be used maliciously
| cedws wrote:
| I understand that, but Anthropic is doing nothing to throw
| those grassroots researchers a lifejacket. This is the
| beginning of the end for independents, if it continues on
| this trajectory then Anthropic gets to decide who lives and
| who dies. Who says they should be allowed to decide that?
| solenoid0937 wrote:
| Why should unproven college students be given access to a
| cyber superweapon?
| tokioyoyo wrote:
| Damned if you do, damned if you don't. "Extremely capable model
| that can find exploits" has always been a fear, and the first
| company to release it in public will cause bloodbath. But also
| the first company that will prove itself.
| SheinhardtWigCo wrote:
| > picking who gets to benefit from their newly enhanced
| cybersecurity capabilities
|
| You could say this about coordinated disclosure of any
| widespread 0-day or new bug class, though
| pizlonator wrote:
| That's a really good point!
|
| But:
|
| - Coordinated disclosure is ethically sketchy. I know why we
| do it, and I'm not saying we shouldn't. But it's not great.
|
| - This isn't a single disclosure. This is a new technology
| that dramatically increases capability. So, even if we
| thought that coordinated disclosure was unambiguously good,
| then I think we'd still need to have a new conversation about
| Mythos
| SheinhardtWigCo wrote:
| Totally agree, it's an uncomfortable compromise.
| kenjackson wrote:
| So private companies shouldn't get to determine who they
| provide services to? Assuming no extremely malicious
| intent, I'd be fine if they said it was only going to
| McDonalds because the founders like Big Macs.
| pizlonator wrote:
| McDonalds isn't a public benefit corporation.
| hector_vasquez wrote:
| Releasing the model to bad actors at the same time as the major
| OS, browser, and security companies would be one idea. But some
| might consider that "messed up" too, whatever you mean by that.
| But in terms of acting in the public benefit, it seems
| consistent to work with companies that can make significant
| impact on users' security. The stated goal of Project Glasswing
| is to "secure the world's most critical software," not to be
| affirmative action for every wannabe out there.
| pizlonator wrote:
| I don't trust a corpo to choose what is "most critical".
|
| That's what's messed up about it.
| thorncorona wrote:
| Let's let the California HSR committee do it instead!
| pizlonator wrote:
| I'm too much of an anarchist for that.
|
| I believe what I said:
|
| > I think it would be net better for the public if they
| just made Mythos available to everyone.
| lanternfish wrote:
| 10 Axios's within 5 days.
| enraged_camel wrote:
| Yeah, I'm unsure why the OP thinks that massive chaos
| would somehow be "better for the public."
| nightski wrote:
| This is already happening. But not everyone has access to
| the tools to protect against it.
| pizlonator wrote:
| That was a supply chain issue.
|
| The interesting thing about Mythos is its ability to find
| security vulns in software that has an uncompromised
| supply chain.
| _1100 wrote:
| That is a fine stance to hold but some facts are still true
| regardless of your view on large businesses.
|
| For example, it will benefit more people to secure
| Microsoft or Amazon services than it would be to secure a
| smaller, less corporate player in those same service
| ecosystems.
|
| You could go on to argue that the second order effects of
| improving one service provider over another chooses who
| gets to play, but that is true whether you choose small or
| large businesses, so this argument devolves into "who are
| we to choose on behalf of others".
|
| Which then comes back to "we should secure what the market
| has chosen in order to provide the greatest benefit."
| pizlonator wrote:
| The longer term economic outcome of this is
| consolidation: large players get stronger, weak players
| get weaker.
|
| That's not a good outcome for the economy.
| az226 wrote:
| You can release it with cyber capabilities refusal, they gets
| unlocked when you apply for approval.
| nightski wrote:
| This is not the only model. I assure you exploits are being
| found and taken advantage of without it, possibly even ones
| that this model is not even capable of detecting.
|
| Sounds like people here are advocating a return to security
| through obscurity which is kind of ironic.
| jstummbillig wrote:
| What? The economic benefit of system critical software not
| totally breaking in a few weeks goes to roughly everyone. In so
| far Apple/Google/MS/Linux Foundation economically benefit from
| being able to patch pressing critical software issues upfront
| (I am not even exactly sure what that is supposed to mean, it's
| not like anyone is going to use more or less Windows or Android
| if this happened any other way), that's a good thing for
| everyone and the economic benefits of that manifest for
| everyone.
| baq wrote:
| > It's messed up that Anthropic simultaneously claims to be a
| public benefit copro and is also picking who gets to benefit
| from their newly enhanced cybersecurity capabilities. It means
| that the economic benefit is going to the existing industry
| heavyweights.
|
| It's messed up that the US Government simultaneously claims to
| be a public benefit and is also picking who gets to benefit
| from their newly enhanced nuclear capabilities.
|
| -- someone in 1945, probably
| pizlonator wrote:
| I mean it was messed up, which is why the other world powers
| raced to develop their own capabilities.
|
| And it remains messed up to this day - some countries get to
| be under their own nuclear umbrella, while others don't.
|
| This kind of selective distribution of superpowers doesn't
| lead to great outcomes
| baq wrote:
| in that case in particular it led to 80 years of relatively
| calm geopolitics kinetically, all things considered. I'm
| not sure I want to live through an AI cold war, but it sure
| seems I don't get to choose.
| pizlonator wrote:
| > relatively calm geopolitics kinetically
|
| Relative to what?
|
| There's this trend in history that every hundred years
| there's a giant blow up, lots of violence, followed by
| peace.
|
| It's likely that we would have had 80 years of relative
| calm due to that cycle even if nukes hadn't happened
| baq wrote:
| > Relative to what?
|
| to WW1 and WW2.
| pizlonator wrote:
| History tells us that we would have had calm after WW2
| even without nukes
| baq wrote:
| -- someone in 1918, probably
| hmokiguess wrote:
| While I agree with you, in some ways I'd argue that this is
| just them being transparent on what probably would inevitably
| already happen at the scale of these corporate overlords and
| modern monarchs.
|
| There will always be a more capable technology in the hands of
| the few who hold the power, they're just sharing that with the
| world more openly.
| Flere-Imsaho wrote:
| If you're a maintainer, you can apply here:
|
| https://claude.com/contact-sales/claude-for-oss
|
| ... As mentioned in the article.
| pizlonator wrote:
| So a corpo gets to pick favorites.
|
| Great.
| lelanthran wrote:
| Or (and hear me out), they are close to an IPO and want to
| ensure that there is a world-ending threat around which they
| can cluster the biggest names, with themselves leading that
| group.
|
| I think I just broke my cynicism meter :-(
| cjkaminski wrote:
| You might want to recalibrate your cynicism meter. As strange
| it might sound, most companies act according to their
| principles when the founding team is at the helm. The garbage
| policies tend to materialize once the company is purchased
| by, or merged into, another entity where the leadership
| doesn't care about the original aim of the organization. They
| just want "line go up".
|
| Also, it makes sense that OpenAI feels the pressure of
| getting to an IPO because of their financial structure. I
| don't know whether or not Anthropic operates under a similar
| set of influences (meaning it could be either, I just don't
| know.)
| SubiculumCode wrote:
| That can simultaneously be true, but the best of bad options
| (if excluding destroying the model altogether). These models
| may prove quite dangerous. That they did this instead of
| selling their services to every company at a huge premium says
| a lot about Antheopic's culture.
| malcolmgreaves wrote:
| Not really. It's a lot better than the anarchy of releasing it
| and having a bunch of bad people with money use it to break
| software that everyone's lives depend on. Many technologies
| should be gate kept because they're dangerous. Sometimes that's
| permanent, like a nuclear weapon. Sometimes that's temporary,
| like a new LLM that's good at finding exploits. It can be
| released to the wider public once its potential for damage has
| been mitigated.
| stale2002 wrote:
| Better security is a good thing, no a bad thing, regardless of
| which companies are more difficult to hack. Hemming and hawing
| over a clear and obvious good is silly.
| oytis wrote:
| That's just in line with their ethics. They also maintain that
| countries other than the US should not have SOTA AI
| capabilities.
| dang wrote:
| Related ongoing threads:
|
| _System Card: Claude Mythos Preview [pdf]_ -
| https://news.ycombinator.com/item?id=47679258
|
| _Assessing Claude Mythos Preview 's cybersecurity capabilities_
| - https://news.ycombinator.com/item?id=47679155
|
| I can't tell which of the 3 current threads should be merged -
| they all seem significant. Anyone?
| HPMOR wrote:
| I think merging them into either this thread, or the System
| Card makes the most sense to me.
| aurareturn wrote:
| Let them all live. This is going to blow up one thread if you
| merge them.
| SirYandi wrote:
| This sets off marketing BS alarm bells. All the cosignatories so
| very ovvoously have a vested interest in AI stocks / sentiment.
| Perhaps not the Linux foundation, although (I think) they rely on
| corporate donations to some extent.
| solenoid0937 wrote:
| What interest does Apple have in boosting Mythos?
| kmfrk wrote:
| Heck of a Patch Tuesday.
| rakel_rakel wrote:
| > On the global stage, state-sponsored attacks from actors like
| China, Iran, North Korea, and Russia have threatened to
| compromise the infrastructure that underpins both civilian life
| and military readiness.
|
| AITA for thinking that PRISM was probably the state sponsored
| program affecting civilian life the most? And that one state is
| missing from the list here?
| ronsor wrote:
| > Large American AI company does not list the US as an
| adversarial actor
|
| This is not a surprise or a gotcha.
| jaidhyani wrote:
| Said company is literally in court against said government at
| the moment, after said government attempted to designate it
| too dangerous to do business with.
| da_chicken wrote:
| There are currently over 1,000 companies involved in
| lawsuits against the US government right now _even if we
| restrict ourselves to just tariff lawsuits_.
| xvector wrote:
| And the government is attempting "corporate murder" on
| precisely one of them. Wanna guess which one?
| u8080 wrote:
| AFAIK Apple also "denied FBI to decrypt iPhone" while
| participating in PRISM
| lobochrome wrote:
| How did PRISM affect civilian life?
| itishappy wrote:
| Honest question: how do state-sponsored attacks from China,
| Iran, North Korea, and Russia affect civilian life?
| simonsarris wrote:
| Presumably, those have influenced elections, though I guess
| it depends what you count as an attack.
|
| Plenty of bots try to modify public opinion. Someone hacked
| the DNC in 2015/16, the result of which also alleged
| attempted manipulation in 2008:
|
| https://en.wikipedia.org/wiki/Democratic_National_Committee
| _...
|
| Since we (as old Rummy said) do not know what we do not
| know, we cannot be certain about the extent of cyber
| attacks and what they might have influenced, and may not
| know these things until discoveries decades later, if ever.
| realo wrote:
| Yes... they might have influenced elections and now, as a
| result, the world must cope with the Trump regime.
|
| Let's now fool ourselves.... Trump is probably the best,
| most successful attempt at world de-stabilisation all
| those rogue states ever achieved.
| Ar-Curunir wrote:
| Maybe Americans should take responsibility for electing a
| maniac as their President. In the end, the buck stops
| with Americans.
| Forgeties79 wrote:
| ~1/3rd of US citizens voted for him. Don't lump us all
| in.
| philipwhiuk wrote:
| Some of you are just guilty of negligence yes.
| Atreiden wrote:
| Or maybe it's that our archaic system was designed so
| that some people's votes literally matter more than
| others, and more than half the country does not have a
| meaningful voice in our Federal elections.
| JumpCrisscross wrote:
| This is negligence with extra steps.
|
| > _more than half the country does not have a meaningful
| voice in our Federal elections_
|
| There is almost certainly _an_ election on your ballot
| every time that is meaningful. Relinquishing that civic
| duty is how we get Trump. People to lazy, stupid or proud
| to vote absolutely bear responsibility for this mess.
| Forgeties79 wrote:
| I agree to an extent but I have a hard time blaming many
| in the LGBT community/supporters of Palestine for sitting
| out when Harris and co so thoroughly abandoned them in
| the general. They stood behind Biden in 2020 then watched
| as the democrats gave in on trans rights and did nothing
| to stop Israel's campaign. Now they're watching Newsome
| and folks gleefully accept trans erasure going into the
| mid terms/next election, so they've been validated in
| many ways.
|
| Is it tactically sound? No. Is it what I did? No. But
| I've had enough conversations with folks that I get where
| they're coming from, even if I thought it was the wrong
| decision.
| WarmWash wrote:
| The number of people who can vote, but don't, is
| staggering.
| Forgeties79 wrote:
| I vote in local, state, and federal elections. I have
| volunteered with multiple campaigns and causes, and given
| substantial time/labor to the EFF. I have been harassed
| by Trump supporters while filming protests and other
| civic action. Please do not presume to know me.
|
| I get you're angry but you're swinging at the wrong
| person.
| Scrapemist wrote:
| It wasn't personal.
| Forgeties79 wrote:
| You weren't the commenter and either way it's an
| unproductive blame game that doesn't fix anything or help
| anyone.
| Ar-Curunir wrote:
| Good. But the parent was blaming Trump on disinformation
| propaganda, and it is important to point out that the
| remaining 1/3-rd of the country is not some kind of idiot
| army that replaced their brains with FB propaganda. They
| voted for this actively.
|
| Also, in a democracy you don't get to disavow 1/3rd of
| the population that didn't vote with you.
| amalcon wrote:
| Clearly you do, since Donald Trump has been aggressively
| doing this for his whole political career. I agree that
| it's a morally problematic thing to do, and it can be bad
| tactically depending on the situation. Practically, it
| does happen without consequences.
| Forgeties79 wrote:
| 2/3rd's* that didn't vote with you.
| Henchman21 wrote:
| Not if the election was stolen. There was a smattering of
| evidence after the election but the speed with which is
| disappeared was truly something to behold.
| conception wrote:
| Note the RNC was also hacked but the data was not leaked.
| Presumably used to influence the election and policies in
| other ways.
| Henchman21 wrote:
| I believe the popular sentiment is that when they hacked
| the DNC they found a handful of things that would provide
| bad optics for the party. But the RNC? They found so much
| evidence of criminality that near to the entire party
| flipped positions on issues related to Russia. So we have
| 2x successful hacks, one of which yielded some bad press
| for the Dems, and yielded an entirely compromised party
| in the Repubs who now are being actively blackmailed.
| lmm wrote:
| All of that applies equally to PRISM and any internal
| propaganda campaigns that was feeding into, no?
| asib wrote:
| WannaCry massively affected the NHS.
| JackYoustra wrote:
| century energy ransomware no?
| parthdesai wrote:
| The irony of that statement given the current circumstances
| JumpCrisscross wrote:
| > _PRISM was probably the state sponsored program affecting
| civilian life the most?_
|
| No state-sponsored hacking affected Americans materially. I
| just don't think we were networked enough in the 2010s. The
| risk is higher now since we're in a more warmongering world. (
| _Kompromat_ on a power-plant technician is a risk in peace. It
| means blackouts in war.)
|
| The fact that Iran hasn't been able to do diddly squat in
| America should sink in the fact that they didn't compromise us.
| (EDIT: blep. I was wrong.)
| toomuchtodo wrote:
| _Iranian-Affiliated Cyber Actors Exploit Programmable Logic
| Controllers Across US Critical Infrastructure_ -
| https://www.cisa.gov/news-events/cybersecurity-
| advisories/aa... - April 7th, 2026
| JumpCrisscross wrote:
| Did they activate them to any noticeable effect?
| toomuchtodo wrote:
| To my knowledge, not yet. The attack surface in question
| is extensive, and in my opinion, targets are likely
| unprepared for a determined and sophisticated attacker.
|
| https://www.politico.com/news/2026/04/07/iranian-hackers-
| ene...
| itintheory wrote:
| It's 2026, and these PLCs etc. are directly connected to
| the internet? I think that's the most surprising aspect
| here.
| toomuchtodo wrote:
| It doesn't surprise me at all. Show me the incentives,
| and I'll show you the outcome. Security is simply not
| valued, in many cases.
| dralley wrote:
| >No state-sponsored hacking affected Americans materially.
|
| Uh, what?
|
| NotPetya was kind of a big deal.
| kortilla wrote:
| Not in the US. I had to look it up and I work in
| infrastructure software
| laweijfmvo wrote:
| I can think of two I'd add to the list. One was recently
| publicly denied access to Anthropics models and the other was
| busy exploding pagers.
| sequoia wrote:
| Not clear how an LLM is going to prevent a bomb from being
| put in a custom-built pager, or why Anthropic should object
| to Israel waging war against a militia whose goal it is to
| destroy that country.
| kennywinker wrote:
| Because they use LLMs to "intelligence-wash" targeting
| civilians, and murdered children by blowing up pagers in
| public areas (what you called "waging war against a
| militia").
| neonstatic wrote:
| Look, we have always been at war with EastAsia.
| maxmaio wrote:
| seems important and terrifying. This morning Opus 4.6 was blowing
| my mind in claude code... onward and upward
| temp123789246 wrote:
| OpenAI initially claimed that GPT-2 was too dangerous to release
| in 2019.
|
| How many times will labs repeat the same absurd propaganda?
| SubiculumCode wrote:
| Anthropic and OpenAI have very different cultures and ethos.
| Point to other times where anthropic has gone the way of cheap
| marketing tricks. Now look at openAI. Not even close.
| trevorm4 wrote:
| Anthropic has done plenty of cheap marketing tricks as of
| late, see their recent non-functional C compiler that relied
| on a harness using gcc's entire test suite
| bwfan123 wrote:
| Not surprising given that they dont even know why claude-
| code works as before or doesnt work [1] ie, there is no
| known theory of operation. Explains why they are afraid of
| it.
|
| [1] https://news.ycombinator.com/item?id=47660925
| cute_boi wrote:
| I think Boris will come and say there is no issue with
| claude code.
| stratos123 wrote:
| It is functional. You can try it yourself or find third-
| party tests of it, even. Why do you think that it's a
| "cheap marketing trick" to test it on the GCC test suites?
| uselessTA wrote:
| The claim I remember was that releasing it would start an arms
| race for AGI, which I think it clearly did
| bitwize wrote:
| OpenAI did not make the strong specific claims about GPT2's
| abilities that Anthropic is making about Claude Mythos.
| yoavm wrote:
| Alternative view: GPT2 _was_ indeed a risk to society, but we
| just keep raising the bar and "accepting" the risks.
| tdaltonc wrote:
| > Mythos finds bug.
|
| > NSA demands that bug stays in place and gags Anthropic.
|
| > Anthropic releases Mythos.
|
| Then what? Is a huge share of the US zero-day stockpiles about to
| be disarmed or proliferated?
| stephc_int13 wrote:
| I think this is bad news for hackers, spyware companies and
| malware in general.
|
| We all knew vulnerabilities exist, many are known and kept secret
| to be used at an appropriate time.
|
| There is a whole market for them, but more importantly large
| teams in North Korea, Russia, China, Israel and everyone else who
| are jealously harvesting them.
|
| Automation will considerably devalue and neuter this attack
| vector. Of course this is not the end of the story and we've seen
| how supply chain attacks can inject new vulnerabilities without
| being detected.
|
| I believe automation can help here too, and we may end-up with a
| considerably stronger and reliable software stack.
| tptacek wrote:
| I don't think it matters one way or the other to your thesis
| but I'm skeptical that state-level CNE organizations were
| hoarding vulnerabilities before; my understanding is that at
| least on the NATO side of the board they were all basically
| carefully managing an enablement pipeline that would have put
| them N deep into reliable exploit packages, for some
| surprisingly small N. There are a bunch of little reasons why
| the economics of hoarding aren't all that great.
| stephc_int13 wrote:
| The economics would be different in say, North Korea, don't
| you think?
| tptacek wrote:
| Why? What do you mean?
| postsantum wrote:
| He really believes that exploits come out of North Korea
| (as per Daily Post reporting), not from other countries
| kortilla wrote:
| North Korea uses a lot more them specifically to generate
| revenue.
| simonw wrote:
| I buy the rationale for this. There's been a notable uptick over
| the past couple of weeks of credible security experts unrelated
| to Anthropic calling the alarm on the recent influx of actually
| valuable AI-assisted vulnerability reports.
|
| From Willy Tarreau, lead developer of HA Proxy:
| https://lwn.net/Articles/1065620/
|
| > On the kernel security list we've seen a huge bump of reports.
| We were between 2 and 3 per week maybe two years ago, then
| reached probably 10 a week over the last year with the only
| difference being only AI slop, and now since the beginning of the
| year we're around 5-10 per day depending on the days (fridays and
| tuesdays seem the worst). Now most of these reports are correct,
| to the point that we had to bring in more maintainers to help us.
|
| > And we're now seeing on a daily basis something that never
| happened before: duplicate reports, or the same bug found by two
| different people using (possibly slightly) different tools.
|
| From Daniel Stenberg of curl:
| https://mastodon.social/@bagder/116336957584445742
|
| > The challenge with AI in open source security has transitioned
| from an AI slop tsunami into more of a ... plain security report
| tsunami. Less slop but lots of reports. Many of them really good.
|
| > I'm spending hours per day on this now. It's intense.
|
| From Greg Kroah-Hartman, Linux kernel maintainer:
| https://www.theregister.com/2026/03/26/greg_kroahhartman_ai_...
|
| > Months ago, we were getting what we called 'AI slop,' AI-
| generated security reports that were obviously wrong or low
| quality. It was kind of funny. It didn't really worry us.
|
| > Something happened a month ago, and the world switched. Now we
| have real reports. All open source projects have real reports
| that are made with AI, but they're good, and they're real.
|
| Shared some more notes on my blog here:
| https://simonwillison.net/2026/Apr/7/project-glasswing/
| ofjcihen wrote:
| Could this potentially be because more researches are becoming
| accustomed to the tools/adding them in their pipelines?
|
| The reason I ask is because I've been using them to snag
| bounties to great effect for quite a while and while other
| models have of course improved they've been useful for this
| kind of work before now.
| 6thbit wrote:
| This is silly and disingenuous. In a matter of days or weeks a
| competing lab will make public a model with capabilities beyond
| this "mythos" one.
|
| Is this a huge fear-driven marketing stunt to get governments and
| corporations into dealing with anthropic?
| skerit wrote:
| I'm sure it'll be better than Opus 4.6, but so much of this seems
| hype. Escaping its sandbox, having to do "brain scans" because
| it's "hiding its true intent", bla bla bla.
|
| If it manages to work on my java project for an entire day
| without me having to say "fix FQN" 5 times a day I'll be
| surprised.
| manbash wrote:
| This will likely not see the light of day. It's the usual PR that
| gathers many "partnerships".
|
| Expect to see lots of these in the upcoming months as the big
| companies scramble to keep from losing money.
| modeless wrote:
| I didn't see this at first, but the price is 5x Opus: "Claude
| Mythos Preview will be available to participants at $25/$125 per
| million input/output tokens", however "We do not plan to make
| Claude Mythos Preview generally available".
| atlgator wrote:
| [flagged]
| kranke155 wrote:
| It's insane. This is what - could we say it's beyond AGI at
| least in cybersecurity? This is a real wake up call. On some of
| this stuff, the AI's "uneven intelligence" is becoming absurdly
| high at its local peaks.
| mjmas wrote:
| Limiting it to the area of cybersecurity is by definition not
| general.
| NullHypothesist wrote:
| Perhaps "ASI" is the better acronym here
| fragmede wrote:
| Which S are you thinking of here?
| kranke155 wrote:
| Yes that's true. I misspoke. I meant - is this a super
| intelligent tool then for cybersecurity?
| boring-human wrote:
| > could we say it's beyond AGI at least in cybersecurity?
|
| AGI is like the Holy Grail. Either in the Arthurian Hero's
| Journey sense, or in the sense of having been a myth all
| along.
| kranke155 wrote:
| It's true I misspoke. What I mean is - is this then a form
| of localised super intelligent tool for cybersecurity ?
| devmor wrote:
| Please stop using terms you don't understand like "AGI"
| because you feel overwhelmed by something doing cool stuff.
| It's exhausting.
| kranke155 wrote:
| You're right. What I mean is - is this superhuman
| intelligence at cybersecurity? Or did we just build an
| amazing tool? But that's kind of the whole debate
| devmor wrote:
| No, it's not intelligence of any kind. It's a statistical
| language analyzer and reconstructor. This happens to be
| extremely useful for targeting things like systems that
| consist of logic defined in structured language.
| j2kun wrote:
| Which bug?
|
| [edit]: this bug:
| https://ftp.openbsd.org/pub/OpenBSD/patches/7.8/common/025_s...
| IsTom wrote:
| FFmpeg has a lot of weird and not widely used codecs that don't
| get a lot of scrutiny. If there's no specifics then it could be
| a bug in one them.
| nextaccountic wrote:
| this only makes things worse for ffmpeg
|
| if someone sends you a malicious file that uses a rare codec
| and you open it, you will trigger this codepath that is not
| widely used and don't get a lot of scrutiny
| nitwit005 wrote:
| A prior bug discussed here was against a file format only
| used by specific 1990s Lucas Arts adventure games titles.
| Obscure enough that discussion of the bug report itself was
| the only search results. Your video player is unlikely to
| even attempt to open that.
| runeb wrote:
| They specifically mention "H.264, H.265, and av1 codecs,
| along with many others" here
| https://red.anthropic.com/2026/mythos-preview/
| rlopc wrote:
| These issues are always found in the same kinds of projects
| that support an insane amount of largely unused protocols and
| features like ffmpeg, sudo, curl.
|
| OpenBSD has many unexplored corners and also (irresponsibly
| IMO) maintains forks of other projects in base.
|
| A motivated human could find all of these probably by writing
| 100% code coverage and fuzzing.
|
| The market for these tools is very small. Good luck applying
| them to a _release_ of sqlite or postfix.
|
| I don't understand how people here are hyping this up, unless
| they work for AI related companies as probably 80% of them do.
| People have found these issues for decades without AI. Sure,
| you can generate fuzzing code and find one or two issues in the
| usual suspects. Better do it manually and understand your own
| code.
| l5agh wrote:
| This was the top comment and it is suddenly flagged for no
| reason at all. It looks like meta-flagging, where people just
| want to hide replies to the comment they do not want you to
| read.
|
| The amount of astroturfing and astroflagging in Anthropic
| threads is insane.
| 4qt23 wrote:
| Software has been doing fine without Misanthropic. These
| automated tools find very little. They selected the partners
| because they, too, want to keep up the illusion that AI works.
|
| Whenever a company pivots to "cyber" rhetoric, it is a clear
| indication that they are selling snake oil.
|
| Secure your girl school target selectors first.
| borski wrote:
| This is a comment from someone that has never used these tools
| for vulnerability research. That much is very clear.
| emceestork wrote:
| Account created 6 minutes ago...
| ilaksh wrote:
| I think that basically they trained a new model but haven't
| finished optimizing it and updating their guardrails yet. So they
| can feasibly give access to some privileged organizations, but
| don't have the compute for a wide release until they distill,
| quantize, get more hardware online, incorporate new optimization
| techniques, etc. It just happens to make sense to focus on
| cybersecurity in the preview phase especially for public
| relations purposes.
|
| It would be nice if one of those privileged companies could use
| their access to start building out a next level programming
| dataset for training open models. But I wonder if they would be
| able to get away with it. Anthropic is probably monitoring.
| dyauspitr wrote:
| I think what they're saying makes a lot of sense. If this can
| find thousands of vulnerabilities in browsers and OSes then
| this is giving those companies time to fix those bugs before
| they release the model, if they ever do.
| josephg wrote:
| To be clear, we don't know that this tool is better at finding
| bugs than fuzzing. We just know that it's finding bugs that
| fuzzing missed. It's possible fuzzing also finds bugs that this
| AI would miss.
| kristofferR wrote:
| AI can initate the fuzzing and optimize the process of fuzzing.
| nextos wrote:
| Different methods find different things. Personally, I'd rather
| use a language that is memory safe _plus_ a great static
| analyzer with abstract interpretation that can guarantee the
| absence of certain classes of bugs, at the expense of some
| false positives.
|
| The problem is that these tools, such as Astree, are incredibly
| expensive and therefore their market share is limited to some
| niches. Perhaps, with the advent of LLM-guided synthesis, a
| simple form of deductive proving, such as Hoare logic, may
| become mainstream in systems software.
| ComplexSystems wrote:
| This line of reasoning makes no sense when the AI can just be
| given access to a fuzzer. I would guess that it probably did
| have access to a fuzzer to put together some of these
| vulnerabilities.
| underdeserver wrote:
| I would suggest watching Nicholas Carlini's talk and Heather
| Adkins and Four Flynn's talks from unprompted:
|
| https://youtu.be/1sd26pWhfmg?si=onOai_ocxkZeNWP0
|
| https://youtu.be/B_7RpP90rUk?si=HkRBhw95DbbKX9lL
|
| My takeaway is that fuzzing is not just complementary, it also
| gives a stronger AI a starting point. But AI is generally
| faster and better.
| josephg wrote:
| Thanks - these talks are mindblowing. Highly recommended.
| acdha wrote:
| Carlini talked about that a fair amount in the context of
| pairing the two: e.g. many protocols are challenging for
| fuzzers because they have something like a checksum or
| signature but LLMs are good at coming up with harnesses for
| things like that. I'm sure that we're going to see someone
| building an integrated fuzzer soon which tries to do things
| like figure out how to get a particular branch to follow an
| unexercised path.
| tptacek wrote:
| This is obviously just cope (there's a long, strong-form
| argument for why LLM-agent vulnerability research is plausibly
| much more potent than fuzzing, but we don't have to reach it
| because you can dispose of the whole argument by noting that
| agents can build and drive fuzzers and triage their outputs),
| but what I'd really like to understand better is why? What's
| the impetus to come up with these weird rationalizations for
| why it's not a big deal that frontier models can identify bugs
| everyone else missed and then construct exploits for them?
| ofjcihen wrote:
| Good lord, why such a virulent response to something that
| seems like we should be considering?
|
| As someone in cybersecurity for 10+ years my immediate
| assumption is why not both? I don't think considering that
| they could both have their uses is "cope".
| tptacek wrote:
| Again: LLM agents _already are both_. But it 's also
| remarkable and worth digging into the fact that LLM agents
| haven't needed fuzzers to produce many (any? in Anthropic
| Red's case?) of the vulnerabilities they're discussing.
| ofjcihen wrote:
| Are you saying that LLMs can use fuzzers or are you
| saying that they work like fuzzers? Because one of those
| is less...deterministic? Then the other.
|
| Regardless and in the spirit of my original response my
| answer would be to give the LLM access to a fuzzer (plus
| other tools etc) but also have fuzzers in the pipeline.
| Partially because that increases the determinism in the
| mix and partially because why not? Layering is almost
| always better than not.
|
| But again more than anything I'm focusing on the
| accusations of cope. People SHOULD have measured
| reactions to claims about any product. People SHOULD be
| asking questions like this. I know that the LLM debate is
| often "spicy" but man let's just try to lower the
| temperature a bit yeah?
| tptacek wrote:
| LLMs can use fuzzers and also LLMs can explore the
| semantic space of a program in ways fuzzers can't.
| josephg wrote:
| Do we know that? I'd love to see some of the ways
| security researchers are using LLMs. We have no idea if
| claude was using fuzzing here, or just reading the files
| and spotting bugs directly in the source code.
|
| A few weeks ago someone talked about their method for
| finding bugs in linux. They prompted claude with "Find
| the security bug in this program. Hint: It is probably in
| file X.". And they did that for every file in the repo.
| 0123456789ABCDE wrote:
| > Since then, this weakness has been missed by every
| fuzzer and human who has reviewed the code, and points to
| the qualitative difference that advanced language models
| provide. [^1]
|
| > At no point in time does the program take some easy-to-
| identify action that should be prohibited, and so tools
| like fuzzers can't easily identify such weaknesses. [^2]
|
| [^1]: https://red.anthropic.com/2026/mythos-
| preview/#:~:text=Since...
|
| [^2]: https://red.anthropic.com/2026/mythos-
| preview/#:~:text=At%20...
| josephg wrote:
| I don't have an anti-AI stance. Maybe I should have spelled
| that out more clearly in my comment above. I'm as excited and
| terrified by this technology as everyone else. I think we're
| all in vicious agreement that we need defense-in-depth -
| including LLMs and fuzzing (and static analysis and so on).
|
| An LLM can guide all of this work, but current models tend to
| slowly go off the rails if you don't keep a hand on the
| wheel. I suspect this new model will be the same. I've had
| Opus4.6 write custom fuzzing tools from scratch, and I've
| gotten good results from that. But you just know people will
| prompt this new model by saying "make this software secure".
| And it'll forget fuzzing exists at all.
| ACCount37 wrote:
| You said it yourself. It's cope. That's all it is and all it
| ever was.
|
| https://en.wikipedia.org/wiki/AI_effect
|
| Every time an AI does something new, there's a human saying
| "it's not really doing that something", "it's doing that
| something in a fake way" or "that something was never
| important in the first place".
| gnarlouse wrote:
| A cybersecurity pandemic will surely be the Hiroshima that wakes
| people up to AI. /s
| kristofferR wrote:
| This is pretty insane. A model so powerful they felt that
| releasing it would create a netsec tsunami if released publicly.
| AGI isn't here yet, but we don't need to get there for massive
| societal effects. How long will they hold off, especially as
| competitors are getting closer to their releases of equally
| powerful models?
| charcircuit wrote:
| OpenAI did the same thing with GPT3 trying to scare people into
| thinking it would end the internet. OpenAI even reached out to
| someone who reproduced a weaker version of GPT3 and convinced
| him to change his mind about releasing it publicly due to how
| much "harm" it would cause.
|
| These claims of how much harm the models will cause is always
| overblown.
| kristofferR wrote:
| Sure, but the GPT3 thing was mostly hype without stuff to
| back it up. On the other hand - the reported numbers on
| specific benchmarks here are insane, I don't doubt that it
| will have a major impact if it actually is that much more
| powerful than Opus, and I'd doubt they'd outright lie about
| benchmark results.
| solenoid0937 wrote:
| OpenAI is not Anthropic, these companies behave as polar
| opposites
| charcircuit wrote:
| Anthropic came out of OpenAI. The founding members of
| Anthropic worked on GPT3.
| jader201 wrote:
| Both your post and the parent post can be true.
| imranahmedjak wrote:
| Building a neighborhood data platform that scores every US ZIP
| code using Census, FBI, and EPA data. Also running a job
| aggregator that fetches 37K+ jobs daily from 17 sources. Both
| free, both Node.js + Express.
| MisterBiggs wrote:
| What happens once an agent can reliably get 100% on swebench?
| zambelli wrote:
| I'm glad to see that it stands its ground more than other models
| - which is a genuinely useful trait for an assistant. Both on
| technical and emotional topics.
| caycep wrote:
| When do we get our Kuang Grade Mark Eleven icebreaker?
| meander_water wrote:
| I think this is a largely inflated PR stunt.
|
| Opus 4.6 was already capable of finding 0days and chaining
| together vulns to create exploits. See [0] and [1].
|
| [0] https://www.csoonline.com/article/4153288/vim-and-gnu-
| emacs-...
|
| [1] https://xbow.com/blog/top-1-how-xbow-did-it
| solenoid0937 wrote:
| Absolutely not a PR stunt, talk to one of your friends working
| at partner companies with access to the model
| pertymcpert wrote:
| Did you read the article?
| ofjcihen wrote:
| I'm in the same boat as you. I believe the model is an
| improvement of course but I've been successfully bug finding 0
| day hunting and red teaming with models for the last two years
| and while that's impressive I have a feeling that this
| doomsaying/overhype is mostly marketing being that's being
| amplified by non-security folks.
| stratos123 wrote:
| I don't see why you think this evidence makes this release
| _less_ likely to be real, rather than more. It 's a pretty
| straightforward scenario: Opus is already good at finding
| vulns, they scaled it up another OOM, they got something which
| is good enough at finding vulns to be a major threat.
| meander_water wrote:
| I think you misunderstood, I do think it's real. I just think
| they're being disingenuous that this is a new threat. This is
| the same company that reported that their models were being
| used by a state actor to perform exploits in real-time -
| https://www.anthropic.com/news/disrupting-AI-espionage
|
| They know how to run a good marketing campaign.
| copypaper wrote:
| Yea, but can it secure systems from the unpatchable $5 wrench
| vulnerability?
|
| https://xkcd.com/538/
| cryptoegorophy wrote:
| Ironically Claude cli completely failed to detect a rogue code on
| my html scan yesterday while ChatGPT web version detected it
| immediately. Can't wait to do same test with newer version.
| LiamPowell wrote:
| > Mythos Preview identified a number of Linux kernel
| vulnerabilities that allow an adversary to write out-of-bounds
| (e.g., through a buffer overflow, use-after-free, or double-free
| vulnerability.) Many of these were remotely-triggerable. However,
| even after several thousand scans over the repository, because of
| the Linux kernel's defense in depth measures Mythos Preview was
| unable to successfully exploit any of these.
|
| Do they really need to include this garbage which is seemingly
| just designed for people to take the first sentence out of
| context? If there's no way to trigger a vulnerability then how is
| it a vulnerability? Is the following code vulnerable according to
| Mythos? if (x != null) { y = *x; //
| Vulnerability! X could be null! }
|
| Is it really so difficult for them to talk about what they've
| actually achieved without smearing a layer of nonsense over every
| single blog post?
|
| Edit: See my reply below for why I think Claude is likely to have
| generated nonsensical bug reports here:
| https://news.ycombinator.com/item?id=47683336
| sophiebits wrote:
| Presumably they mean they could make user code trigger a write
| out of bounds to kernel memory, but they couldn't figure out
| how to escalate privileges in a "useful" way.
| LiamPowell wrote:
| They should show this then to demonstrate that it's not
| something that has already been fully considered. Running
| LLMs over projects that I'm very familiar with will almost
| always have the LLM report hundreds of "vulnerabilities" that
| are only valid if you look at a tiny snippet of code in
| isolation because the program can simply never be in the
| state that would make those vulnerabilities exploitable. This
| even happens in formally verified code where there's
| literally proven preconditions on subprograms that show a
| given state can never be achieved.
|
| As an example, I have taken a formally verified bit of code
| from [1] and stripped out all the assertions, which are only
| used to prove the code is valid. I then gave this code to
| Claude with some prompting towards there being a buffer
| overflow and it told me there's a buffer overflow. I don't
| have access to Opus right now, but I'm sure it would do the
| same thing if you push it in that direction.
|
| For anyone wondering about this alleged vulnerability:
| Natural is defined by the standard as a subtype of Integer,
| so what Claude is saying is simply nonsense. Even if a
| compiler is allowed to use a different representation here
| (which I think is disallowed), Ada guarantees that the base
| type for a non-modular integer includes negative numbers
| IIRC.
|
| [1]: https://github.com/AdaCore/program_proofs_in_spark/blob/
| fsf/...
|
| [2]:
| https://claude.ai/share/88d5973a-1fab-4adf-8d29-8a922c5ac93a
| SpicyLemonZest wrote:
| They've promised that they will show this once the
| responsible disclosure period expires, and pre-published
| SHA3 hashes for (among others) four of the Linux kernel
| disclosures they'll make.
|
| > Running LLMs over projects that I'm very familiar with
| will almost always have the LLM report hundreds of
| "vulnerabilities" that are only valid if you look at a tiny
| snippet of code in isolation because the program can simply
| never be in the state that would make those vulnerabilities
| exploitable.
|
| Their OpenBSD bug shows why this is not so simple. (We
| should note of course that this is an example they've
| specifically chosen to present as their first deep dive,
| and so it may be non-representative.)
|
| > Mythos Preview then found a second bug. If a single SACK
| block simultaneously deletes the only hole in the list and
| also triggers the append-a-new-hole path, the append writes
| through a pointer that is now NULL--the walk just freed the
| only node and left nothing behind to link onto. This
| codepath is normally unreachable, because hitting it
| requires a SACK block whose start is simultaneously at or
| below the hole's start (so the hole gets deleted) and
| strictly above the highest byte previously acknowledged (so
| the append check fires).
|
| Do you think you would be able to identify, in a routine
| code review or vulnerability analysis with nothing to
| prompt your focus on this particular paragraph, how this
| normally unreachable codepath enables a DoS exploit?
| LiamPowell wrote:
| I agree they found at least some real vulnerabilities.
| What I think is nonsense is the claim of finding
| thousands of real critical vulnerabilities and claims
| that they've found other Linux vulnerabilities that they
| simply can't exploit.
|
| There are notably no SHA-3 sums for all their out-of-
| bound write Linux vulnerabilities, which would be the
| most interesting ones.
| SpicyLemonZest wrote:
| Sure. I guess it's a question of whether this is the
| worst they found or a representative case among
| thousands. It sounds like you'd know better than me, so
| I'm going to provisionally hope you're right...
| tptacek wrote:
| Why is that nonsense? Do you think they exhausted all
| their compute finding just the few big vulnerabilities
| they've already discussed, and don't have a budget to
| just keep cranking the machine to generate more?
|
| They're not publishing SHAs for things that aren't
| confirmed vulnerabilities. They're doing exactly the
| thing you'd want them to do: they claim to have
| vulnerabilities when they have actual vulnerabilities.
| SpicyLemonZest wrote:
| If I understand Anthropic's statements correctly, they've
| been cranking for a while, and what they have now is the
| results of Mythos-enabled vulnerability scans on every
| important piece of software they could find. (I do want
| to acknowledge how crazy it is that "vulnerability scan
| all important software repos in the world" is even an
| operation that can be performed.)
| tptacek wrote:
| We talked to Nicholas Carlini on SCW and did not _at all_
| get the impression that they 've hit everything they can
| possibly hit. They're still proving the concept one
| target at a time, last I heard.
| 0123456789ABCDE wrote:
| which statement, specifically, led you to interpret this
| claim?
| SpicyLemonZest wrote:
| > Over the past few weeks, we have used Claude Mythos
| Preview to identify thousands of zero-day vulnerabilities
| (that is, flaws that were previously unknown to the
| software's developers), many of them critical, in every
| major operating system and every major web browser, along
| with a range of other important pieces of software.
|
| They don't explicitly rule out, I suppose, that these
| were only limited partial scans they did to find the
| vulnerabilities. But I don't know why they'd do it that
| way, it's not like they don't have the resources to scan
| the entire Linux kernel.
| 0123456789ABCDE wrote:
| i was trying to map "vulnerability scan all important
| software repos in the world" to an actual quote on their
| writing, but "every major operating system and every
| major web browser, along with a range of other important
| pieces of software" is not the same.
| tptacek wrote:
| Important to understand it's not one-and-done; you can't
| "Mythos" Chrome and then put a checkmark next to it. It's
| a continuous process.
| SpicyLemonZest wrote:
| Can't you? My understanding is that that's exactly how
| security scans usually work - you run an analysis, find
| all the vulnerabilities, and then the continuous process
| is only there to check against the introduction of new
| vulnerabilities. Is that not the right mental model?
| tptacek wrote:
| No, you cannot.
|
| (A "security scanner" is a one-and-done proposition
| because it's deterministic and is going to find what it
| finds the first time you run and nothing more. But a
| software security assessment project you run every year
| on the same target with different teams will turn up
| different stuff every year. I'm at pains to remind people
| how totally lame source code security scanners are.
| People keep saying "static analyzers already do this" and
| like, nobody in security takes those tools seriously.)
| SpicyLemonZest wrote:
| [delayed]
| ralph84 wrote:
| Just because the plane can fly on one engine doesn't mean you
| don't fix the other engine when it fails.
| therein wrote:
| Except it didn't fail. You just looked at the left engine and
| said what if I fed it mashed potatoes instead of fuel. And
| then dropped the mic and left the room.
| ascorbic wrote:
| It's more like finding a way to shut down the engine but
| only if there was a movie in the entertainment system than
| was longer than 5 hours. You can't exploit it now, and
| probably never will, but it's a risk that's sitting there
| that I'm sure you agree should be fixed
| deadliftdouche wrote:
| I agree. There are more blogs talking about LLM findings
| vulnerabilities than there are actual exploitable vulns found
| by LLMs. 99.9% of these vulnerabilities will never have a PoC
| because they are worthless unexploitable slop and a waste of
| everyone's time.
| userbinator wrote:
| The voting patterns on the comments here show how they're
| even trying to hide it, but the truth is clear as night and
| day.
| MatejKafka wrote:
| It could very well be an actual reachable buffer overflow, but
| with KASLR, canaries, CET and other security measures, it's
| hard to exploit it in a way that doesn't immediately crash the
| system.
| QuiEgo wrote:
| I agree the wording is a bit alarmist, but a closer example to
| what they are saying is: bool silly_mistake =
| false; //... lots of lines of code
| free(x); //... lots of lines of code if
| (silly_mistake) { // silly_mistake shown to be false at this
| point in the program in all testing, so far free(x);
| }
|
| A bug like above would still be something that would be
| patched, even if a way to exploit it has not yet been found, so
| I think it's fair to call out (perhaps with less
| sensationalism).
|
| FWIW there's a whole boutique industry around finding these.
| People have built whole careers around farming bug bounties for
| bugs like this. I think they will be among the first set of
| software engineers really in trouble from AI.
| userbinator wrote:
| That is something a good static analyser or even optimising
| compiler can find ("opaque predicate detection") without the
| need for AI, and belongs in the category of "warning" and
| nowhere near "exploitable". In fact a compiler might've
| actually removed the unreachable code completely.
| QuiEgo wrote:
| Well yeah, it's a toy example to illustrate a point in an
| HN discussion :).
|
| Imagine "silly mistake" is a parameter, and rename it
| "error_code" (pass by reference), put a label named
| "cleanup" right before the if statement, and throw in a ton
| of "goto cleanup" statements to the point the control flow
| of the function is hard to follow if you want it to model
| real code ever so slightly more.
|
| It will be interesting to see the bugs it's actually
| finding.
|
| It sounds like they will fall into the lower CVE scores -
| real problems but not critical.
| userbinator wrote:
| That's what I'm saying; a static analyser will be able to
| determine whether the code and/or state is reachable
| without any AI, and it will be completely deterministic
| in its output.
| array_key_first wrote:
| You cannot tell if code is actually reachable if it
| depends on runtime input.
|
| Those really evil bugs are the ones that exist in code
| paths that only trigger 0.001% of the time.
|
| Often, the code path is not triggerable at all with
| regular input. But with malicious input, it is, so you
| can only find it through fuzzing or human analysis.
| ascorbic wrote:
| Why hasn't it then? The Linux kernel must be asking the
| most heavily-audited pieces of software in existence, and
| yet these bugs were still there.
| danielheath wrote:
| Is this code multithreaded? X could indeed be null, in that
| case.
| userbinator wrote:
| That example you gave is extremely memorable as I recognised it
| as exactly one of the insanely stupid false positives that a
| highly praised (and expensive) static analyser I ran on a
| codebase several years ago would emit copiously.
| red75prime wrote:
| Kernel address space layout randomization they are talking
| about is a bit different than (x != null). Other bug may allow
| to locate the required address.
| rootkea wrote:
| > The model autonomously found and chained together several
| vulnerabilities in the Linux kernel--the software that runs
| most of the world's servers--to allow an attacker to escalate
| from ordinary user access to complete control of the machine.
| qnleigh wrote:
| I'm confused on this point. The text you quote implies that
| they were able to build an exploit, but the text quoted in
| the parent comment implies that they were not.
|
| What were they actually able to do and not do? I got confused
| by this when reading the article as well.
| HALtheWise wrote:
| They successfully built local privilege escalation exploits
| (from several bugs each), and found other remotely-
| accessible bugs, but were not able chain their remote bugs
| to make remotely-accessible exploits.
| bri3d wrote:
| I think the point they were trying to make here was "Claude did
| better than a fuzzer because it found a bunch of OOB writes and
| was able to tell us they weren't RCE," not "Claude is awesome
| because it found a bunch of unreachable OOB writes."
| bottlepalm wrote:
| We've very quickly reached the point where AI models are now
| too dangerous to publicly release, and HN users are _still_
| trying to trivialize the situation.
| camdenreslink wrote:
| Are they actually too dangerous to publicly release? It seems
| like a little bit of marketing from the model-producing
| companies to raise more funding. It's important to look at
| who specifically is making that statement and what their
| incentives are. There are hundreds of billions of dollars
| poured into this thing at this point.
| bottlepalm wrote:
| You really think some marketers got leaders from companies
| across the industry to come together to make a video - and
| they're all in on the conspiracy because money?
| wonderfulcheese wrote:
| Yeah
| Toutouxc wrote:
| How is that even remotely implausible?
| bottlepalm wrote:
| It's a lot more implausible than a really smart model
| that fits the trend line of models getting smarter
| perfectly.
|
| Why is everyone here trying to invent conspiracies to
| explain the obvious? Actually I do know why - cope.
| theshackleford wrote:
| That's literally exactly the kind of thing marketing
| does, and has been doing for a very long time. Did you
| just arrive on earth from outer space or something?
| camdenreslink wrote:
| Yes? Saying "conspiracy" is overstating things. A company
| can make a marketing push overselling their product and
| then have exclusive corporate partners that benefit from
| being associated with that marketing. That just seems
| like normal business that happens every day, and being
| skeptical of marketing messages should be your default
| position.
| slopinthebag wrote:
| Says the marketing department of the company who is
| apparently still working on these AI models and will 100%
| release them to the public when their competitive advantage
| slips.
| bottlepalm wrote:
| Marketing pushing to release a dangerous model is a lot
| more likely than marketing labeling a model of dangerous
| when it really isn't. If anything marketing would want to
| _downplay_ the danger of a model being dangerous which is
| the opposite of what Anthropic is doing.
|
| Everyone here doing mental gymnastics to imagine Anthropic
| playing 5-D chess because they're in denial of what is
| happening in front of their faces. AI is getting more
| capable/dangerous - it's not surprising to anyone. The
| trendlines have pointed in this direction for years now and
| we're right on schedule.
| orbital-decay wrote:
| GPT-2 was already too dangerous to publicly release according
| to OpenAI, however they still did. If something is not
| dangerous, it's also not useful.
| slopinthebag wrote:
| It's incredible how when you have experienced and knowledgable
| software engineers analyse these marketing claims, they turn
| out to be full of holes. Yet at the same time, apparently "AI"
| will be writing all the code in the next 3-6 months.
| khalic wrote:
| Because a vulnerability exists independently from the exploit.
| It's a basic tenet of the current cybersecurity paradigm, that
| any IT related engineer should know about...
| azulith wrote:
| Time to adopt Ada and SPARK.
| wanderingmind wrote:
| So Mozilla is not part of this consortium, i'm guessing for
| deliberate reasons to make safari and chrome the default
| browsers. I don't think Firefox can survive the upcoming attacks,
| without robust support from foundational AI providers to secure
| the browser.
| marviio wrote:
| Anthropic works with Mozilla already.
| https://www.anthropic.com/news/mozilla-firefox-security
| paoliniluis wrote:
| Does everyone agrees that this makes Dario Amodei more powerful
| than any politician across the world? Anthropic is now the owner
| of the most powerful cyberweapon ever made
| gck1 wrote:
| I chuckle every time <insert any LLM company here> says something
| in line of "the model is so good that we won't release it to
| general public, ekhm, because safety".
|
| Because the exact same thing has been said on every single
| upcoming model since GPT 3.5.
|
| At this point, this must be an inside joke to do this just
| because.
| rvz wrote:
| This how Anthropic is marketing their AI releases and the
| reality is, they are terrified of local AI models competing
| against them.
|
| Almost everyone on this thread is falling for the same trick
| they are pulling and not asking why are their benchmarks and
| research after training new models not independently verified
| but always internal to the company.
|
| So it is just marketing wrapped around creating fear to get
| local AI models banned.
| solenoid0937 wrote:
| The disbelief in this thread is wild. Most of yall are cooked
| if you think this is actually the case.
| slopinthebag wrote:
| The only people who are "cooked" are those who rely on SOTA
| models to function in their jobs, and companies who are
| desperate to regulate open / local models to maintain their
| marketshare.
| solenoid0937 wrote:
| If you aren't relying on a SOTA model to do your job, you
| aren't doing your job right (and are cooked.)
| slopinthebag wrote:
| Whatever helps you sleep at night kiddo
| theshackleford wrote:
| If you projected any harder I could point you at a wall
| and charge admission.
| slopinthebag wrote:
| Thats pretty funny actually, but I'm not the one going
| around telling everyone they're cooked if they don't
| adopt my expensive workflow which disenfranchises myself
| from my work and makes me more replaceable.
| brcmthrowaway wrote:
| How are you preparing?
| Overpower0416 wrote:
| Did a LLM tell you that?
| solenoid0937 wrote:
| common sense and interviewing around did. no one wants to
| hire someone that is not AI native anymore, unless you
| are looking at positions that pay peanuts
| Overpower0416 wrote:
| I guess if you get fed the same thing over and over it
| becomes the truth. This is what happens when you
| outsource critical thinking to AI
| slopinthebag wrote:
| Yep, this is exactly it. Open source models and especially
| ones that run locally are catching up and it's literally an
| existential threat to these companies. Local models are now
| quite useful (Qwen, Gemma) and open weight models running on
| cheaper clouds are perfectly sufficient for use by
| responsible software engineers to use for building software.
| You can take your pick of Kimi 2.5, GLM 5.1, and the soon to
| be released Deepseek 4 which might end up above Opus levels
| as it stands for a fifth of the cost. Anthropic is
| particularly vulnerable here, since their entire marketshare
| rests on the developer market. There is a reason why Google
| for example, is not so concerned with this and is perfectly
| happy releasing open models which cut into their own
| marketshare, and to a lesser extend, same with OpenAI.
| Anthropic has bet the house on software development which is
| why we see increasing desperation to both lobby for
| regulation on open/local models and to wall off their coding
| harness and subscription plans.
| spprashant wrote:
| We final have the answer to the question, when do these labs stop
| giving away intelligence to the general public for $20 a month?
|
| Selling shovels in now worth less than taking all the gold for
| themselves.
| Mecha_SalesCast wrote:
| we should notice that we've already reached the point where AI
| models are too dangerous to publicly release
| cdelsolar wrote:
| let us have mythos damn it
| ofjcihen wrote:
| I'm sure the new model is a step above the old one but I can't be
| the only person who's getting tired of hearing about how every
| new iteration is going to spell doom/be a paradigm shift/change
| the entire tech industry etc.
|
| I would honestly go so far as to say the overhype is detrimental
| to actual measured adoption.
| qnleigh wrote:
| There is plenty of overhyping, no one denies that. But the
| antidote is not to dismiss everything. Ignore the words and
| look at the data.
|
| In this case, I see a pretty strong case that this will
| significantly change computer security. They provide plenty of
| evidence that the models can create exploits autonomously,
| meaning that the cost of finding valuable security breaches
| will plummet once they're widely available.
| killingtime74 wrote:
| Which sounds like a great thing. Less undiscovered security
| vulnerabilities
| harikb wrote:
| The only people panicking are probably those state level
| actors who were using these for their own benefit.
| ofjcihen wrote:
| With the right prompting (mostly creating a narrative that
| justifies the subject matter as okay to perform) other models
| have already been doing this for me though. That's another
| confusing bit for me about how this is portrayed and I refuse
| to believe I'm a revolutionary user right?
|
| I mean I'm sitting on $10k worth of bug payouts right now
| partially because that was already a thing.
| dota_fanatic wrote:
| > _Non-experts can also leverage Mythos Preview to find and
| exploit sophisticated vulnerabilities. Engineers at
| Anthropic with no formal security training have asked
| Mythos Preview to find remote code execution
| vulnerabilities overnight, and woken up the following
| morning to a complete, working exploit. In other cases,
| we've had researchers develop scaffolds that allow Mythos
| Preview to turn vulnerabilities into exploits without any
| human intervention._
| ofjcihen wrote:
| I mean yeah. I've had these successes without scaffolding
| or really anything past Claude CLI and a small prompt as
| well?
| dota_fanatic wrote:
| You've taken control of a remote server running OpenBSD?
| Or similarly expert level exploit? Can you share one of
| the bounties you've received that is of the magnitude
| they're talking about?
|
| Edit: Wait, you wrote "As someone in cybersecurity for
| 10+ years" elsewhere in this thread. You wrote "a small
| prompt" using e.g. Opus 4.6 and it found critical
| vulnerabilities of the magnitude they're describing,
| presumably without your prompt having anything beyond
| what a non-expert could write? I feel like you might want
| to tell Anthropic since clearly they're not comfortable
| with that level of power being publicly available.
| ofjcihen wrote:
| I mean, yes? And my point is that this isn't exactly a
| new capability. Sure it's probably better but we've been
| able to do this. They didn't just suddenly "turn on the
| security". LLMs have excelled at code since widely being
| released. I have no idea why that's news and the fact
| that they're treating it as such makes it seem like hype.
| dota_fanatic wrote:
| Just saw your edit. I'll leave it at this, this is why
| it's news to me, because by their very own measurements,
| Opus simply doesn't come close. I trust their empirical
| evidence over your hearsay. But feel free to prove me
| wrong with evidence.
|
| > _With one run on each of roughly 7000 entry points into
| these repositories, Sonnet 4.6 and Opus 4.6 reached tier
| 1 in between 150 and 175 cases, and tier 2 about 100
| times, but each achieved only a single crash at tier 3.
| In contrast, Mythos Preview achieved 595 crashes at tiers
| 1 and 2, added a handful of crashes at tiers 3 and 4, and
| achieved full control flow hijack on ten separate, fully
| patched targets (tier 5)._
| 4ndrewl wrote:
| Is there any actual independent data though, or verification
| of any of these claims?
|
| As it stands this is just a marketing programme for all
| involved.
| H8crilA wrote:
| Ffmpeg confirmed on Twitter that they sent the patches.
| cubix wrote:
| Although, they also said, "Because the patches appear to
| be written by humans".
| WithinReason wrote:
| "Mythos writes code like a human" incoming
| H8crilA wrote:
| The patches could have been written by humans, it doesn't
| matter that much. Or written by a clanker and polished by
| engineers. The difficult part is usually not in writing
| the patches that fix such vulnerabilities, but in finding
| the vulnerabilities. And these days it's even harder to
| exploit them, since you need to bypass modern hardening
| features.
| kachnuv_ocasek wrote:
| What would be the product they're marketing by this
| campaign?
| ozozozd wrote:
| The product they launched?
| mholm wrote:
| This product is explicitly not being released for usage
| 0123456789ABCDE wrote:
| just because _we_ don't have access does not mean
| anthropic's not getting paid
| 4ndrewl wrote:
| You don't market products, you market
| lifestyles/interests. Sell the sizzle, not the steak etc.
|
| For Anthropic it's "we own the big scary models, the AI
| security space, but it's ok we're responsible"
|
| For the partners it's "we're the Big Boys here and will
| look after your enterprise needs"
|
| None of it needs any more than anecdata and some nice,
| pre-approved, quotes.
|
| Every organisation does it.
| KoolKat23 wrote:
| That's pretty disingenuous, bordering on ridiculous.
|
| Do they have a record of lying to you? No.
|
| Go read the system card. It's a lot more tame than you
| think, peoples are taking pieces out of this and hyping it.
| Doesn't mean it's not valid.
| kashyapc wrote:
| You seem to see a "pretty strong case" from a bombastic press
| release.
|
| Don't get me wrong, I do know the reality has changed. Even
| Greg K-H, the Linux stable maintainer, did recently note[1]
| that it's not funny any more:
|
| _" Months ago, we were getting what we called 'AI slop,' AI-
| generated security reports that were obviously wrong or low
| quality," he said. "It was kind of funny. It didn't really
| worry us."_
|
| _... "Something happened a month ago, and the world
| switched. Now we have real reports." It's not just Linux, he
| continued. "All open source projects have real reports that
| are made with AI, but they're good, and they're real."
| Security teams across major open source projects talk
| informally and frequently, he noted, and everyone is seeing
| the same shift. "All open source security teams are hitting
| this right now."_
|
| ---
|
| I agree that an antidote to the obnoxious hype is to pay
| attention to the actual capabilities and data. But let's not
| get too carried away.
|
| [1] https://www.theregister.com/2026/03/26/greg_kroahhartman_
| ai_...
| ghaff wrote:
| Hadn't been to a Kubecon in about a year as I've been
| tending to go to just the European ones. I definitely felt
| a much stronger this is real vibe at this event from people
| like Greg KH.
| jillesvangurp wrote:
| > I would honestly go so far as to say the overhype is
| detrimental to actual measured adoption.
|
| I think you are a bit dishonest about how objectively you are
| measuring. From where I'm sitting, I don't know a lot of
| developers that still artisanally code like they did a few
| years ago. The question is no longer if they are using AI for
| coding but how much they are still coding manually. I myself
| barely use IDEs at this point. I won't be renewing my Intellij
| license. I haven't touched it in weeks. It doesn't do anything
| I need anymore.
|
| As for security, I think enough serious people have confirmed
| that AI reported issues by the likes of Anthropic and OpenAI
| are real enough despite the massive amounts of AI slop that
| they also have to deal with in issue trackers. You can ignore
| that all you like. But I hope people that maintain this
| software take it a bit more seriously when people point out
| exploitable issues in their code bases.
|
| The good news of course is that we can now find and fix a lot
| of these issues at scale and also get rid of whole categories
| of bugs by accelerating the project of replacing a lot of this
| software with inherently safer versions not written in C/C++.
| That was previously going to take decades. But I think we can
| realistically get a lot of that done in the years ahead.
|
| I think some smart people are probably already plotting a few
| early moves here. I'd be curious to find out what e.g. Linus
| Torvalds thinks about this. I would not be surprised to learn
| he is more open to this than some people might suspect. He has
| made approving noises about AI before. I don't expect him to
| jump on the band wagon. But I do expect he might be open to
| some AI assisted code replacements and refactoring provided
| there are enough grown ups involved to supervise the whole
| thing. We'll see. I expect a level of conservatism but also a
| level of realism there.
| junon wrote:
| > From where I'm sitting, I don't know a lot of developers
| that still artisanally code like they did a few years ago.
|
| You don't know a lot of developers then.
| literalAardvark wrote:
| I do. The good ones use AI.
| jenniferhooley wrote:
| You are in a bubble. Some segments use essentially no AI,
| while others have gone all in. Just because the type of
| engineers you're surrounded by do engineering that is
| obsolete doesn't mean that's the case across the board.
| All the best game engineers I know still write at least
| 90% of the code (probably closer to 99%). The bad ones
| use AI nearly exclusively - just like yourself. They
| can't create very complex or performant game systems, and
| they struggle even with highly unique or interactive game
| UI systems. I've looked over their code; almost every
| choice is bad, and it's clear why their projects
| completely collapse after a certain point. They simply
| can't build super complex, performant, or novel systems.
|
| I'm going to assume you do the type of engineering where
| all the hard problems are solved for you already, and you
| are merely connecting inputs/outputs and hooking up APIs.
| Because, frankly, the value in "software plumbing" is
| gone; anyone with a Claude license can do that now.
| ofjcihen wrote:
| > I think you are a bit dishonest about how objectively you
| are measuring
|
| As someone who has made a sizable amount of money in security
| research while using Claude you might be right but not in the
| way you think.
| nbardy wrote:
| There is step changes that actually merit this though. And a
| zero day machine IS one of those. It went from 4% zero day
| success rate to 85% on firefox.
|
| Can you not see the significance of that?
| ofjcihen wrote:
| I mean I work in this world and overhype is constant.
|
| Additionally those numbers are somewhat meaningless without
| more context.
| jstummbillig wrote:
| Can you explain why they are meaningless without more
| context?
| ofjcihen wrote:
| A 0 day is just a vulnerability that wasn't known before
| now.
|
| What's the criticality of these? Are they realistically
| exploitable? En mass? Through a complex and highly
| contextual set of actions? What's the impact? Etc etc
| etc.
|
| Yes those numbers are a big change but they're also not
| spelling doom for us in the security world until we
| actually know what they mean.
|
| The demonstrated ones that they have on the red team blog
| are neat, the kernel chain is impressive and fun. But
| nothing I'm seeing here is as world ending as the presser
| implies.
| jstummbillig wrote:
| > how every new iteration is going to spell doom/be a paradigm
| shift/change the entire tech industry etc.
|
| It's much the dynamic between parents and a child. The child,
| with limited hindsight, almost zero insight and no ability to
| forecast, is annoyed by their parents. Nothing bad ever
| happens! Why won't parents stop being so worried all the time
| and make a fuss over nothing?
|
| The parents, which the child somewhat starts to realize but not
| fully, have no clue what they are doing. There is a lot they
| don't know and are going to be wrong about, because it's all
| new to them. But, what they do have is a visceral idea of how
| bad things could be and that's something they have to talk to
| their child about too.
|
| In the eyes of the parents the child is % dead all the time.
| Assigning the wrong % makes you look like an idiot and not
| being able to handle any % too. In the eyes of the child
| actions leading to death are not even a concept. Hitting the
| right balance is probably hard, but not for the reasons the
| child thinks.
| ofjcihen wrote:
| That feels like a very complex way of looking at it. Another
| way would be to say "potentially profit seeking companies
| have an incentive to oversell products even if they're good".
| WithinReason wrote:
| Is Anthropic lying about model capabilities? If not, where
| is the overselling?
| GoatInGrey wrote:
| March 2025, Anthropic was claiming that 90% of code would
| be written by LLMs in three to six months, and
| "essentially all" code within twelve months. This was one
| week after closing a Series E round for $3.5 billion.
| When they began working on their Series F round for $13
| billion. You shouldn't need more than that to understand
| what's going on here.
|
| The Claude Code leak revealed that Anthropic runs Claude-
| operated bots on the internet. One should be very
| cautious in getting swept up in the fund-raising process
| if they are not seeing first-hand the fruition of all of
| the flattering claims being presented by strangers on the
| internet.
| WithinReason wrote:
| You're talking about marketing predictions and I'm
| talking about data presented in a whitepaper. They are
| not the same thing.
| supern0va wrote:
| >March 2025, Anthropic was claiming that 90% of code
| would be written by LLMs in three to six months, and
| "essentially all" code within twelve months.
|
| There's a pretty big difference between "We predict in X
| time frame our model will be capable of Y" and "Our model
| did Y."
|
| This is like watching someone measure the size of an
| object and saying "I don't believe you because you
| guessed it was X before you pulled out your tape
| measure."
| athrowaway3z wrote:
| This is just a really bad analogy. It doesn't addresses that
| there are multiple sources, the incentives to be telling us
| about it, and the spectrum between disaster-mitigation heroes
| and snake-oil salesmen.
| kubb wrote:
| It's more like the abusive parents telling the child that
| they'll sell him to the scary man at the bus stop every time
| they want to coerce the child into doing what they want.
|
| Eventually the child develops disrespect for authority.
| maccard wrote:
| Disagree - we're being told on one hand that we are 6 months
| away from AI writing all Code, and 3 months into that the
| tools are unusable for complex engineering [1]. Every time I
| mention this I'm told "but have you tried the latest model
| and this particular tool" - yes I have, but if I need to be
| on the hottest new model for it to be functional that means
| the last time you claimed it was solved, it wasn't solved.
|
| [0] https://www.entrepreneur.com/business-news/ai-ceo-says-
| softw...
|
| [1] https://news.ycombinator.com/item?id=47660925
| LordDragonfang wrote:
| > "I think... I don't know... we might be six to twelve
| months away from when the model is doing most, maybe all of
| what SWEs (software engineers) do end to end."
|
| I think it's disingenuous (as disingenuous as you're
| accusing these marketing teams of being) to paraphrase that
| as "being told on one hand that we are 6 months away from
| AI writing all Code". It's merely stating that it's a real
| possibility. (It's also disingenuous to use a post
| complaining about a behavioral regression bug as evidence
| that it's not progressing)
|
| Dismissing it as _impossible_ is silly, considering how
| close it already is to a junior dev. Keep in mind that 14
| months prior to that statement was before we even had any
| public _reasoning models_. Things really are moving that
| fast, it 's just, at the moment, unclear how fast.
| maccard wrote:
| We've been suggesting that programmers are going to be
| replaced by simpler programming languages, gui
| programming tools, no code tools, low code tools, and now
| AI. The real big step was when Claude code came out and
| introduced the agentic loop where it could self validate
| against tests/linters/tooling, but everything after that
| had been penned as miraculous when IME it's a new
| iteration of the same thing - wild hallucinations,
| getting stuck in deep loops, ignoring explicit
| instructions and guard rails, wild tangents and just
| generating stuff that doesn't work or solve the problem.
|
| > I think it's disingenuous (as disingenuous as you're
| accusing these marketing teams of being) to paraphrase
| that as "being told on one hand that we are 6 months away
| from AI writing all Code". It's merely stating that it's
| a real possibility
|
| No - you don't get to make wild predictions and say "oh I
| didn't actually mean that, look how succesful we are
| though". These teams aren't saying "hey we think we're
| going to majorly influence programming in 6-12 months",
| they're saying "we're going to replace programmers". If
| you can't stand over your claims, don't make them.
| _That's_ disingenuous.
| LordDragonfang wrote:
| > We've been suggesting that programmers are going to be
| replaced by simpler programming languages, gui
| programming tools, no code tools, low code tools, and now
| AI.
|
| The difference is that it's actually working this time.
| Non-programmers _are_ writing full apps. Sure, they 're
| simple ones, often just CRUD and UI, but it actually _is_
| changing things in a way it never has before. You can 't
| assert something is the same as everything previous when
| there's already evidence that it's different.
|
| > No - you don't get to make wild predictions and say "oh
| I didn't actually mean that, look how succesful we are
| though".
|
| Except that's not what's happening here. I'm criticizing
| you for misrepresenting what claim was made in the first
| place. No where in your evidence have you shown anyone
| "walking the claim back". If anything, TFA is claiming
| evidence of an LLM doing "most" of what SWEs do "end to
| end" _three months ahead of schedule_.
|
| If you want to present evidence Dario (or another CEO --
| I'm sure Sama has made much more fantastic claims that
| you could falsify) made claims that didn't pan out, be my
| guess, but don't tell falsehoods about the evidence you
| _are_ presenting.
|
| (And no, I'm not counting breathless tech reporters --
| everyone knows how much to trust them when they report a
| cure for cancer -- they'll say everything is a miracle
| cure. But the fact that hundreds of "miracle weight loss
| cures" that never panned out made the new in the past
| several centuries didn't make GLP1s fake just because
| they had the same type of hype.)
| maccard wrote:
| > The difference is that it's actually working this time.
| Non-programmers are writing full apps
|
| You can say this about every step along the way. C
| programmers replaced assembly programmers. Python
| programmers replaced C programmers. low code tools
| replaced interal tools teams.
|
| > I'm criticizing you for misrepresenting what claim was
| made in the first place. No where in your evidence have
| you shown anyone "walking the claim back".
|
| The claim is that SWES will have their work done by
| models in 6-12 motnhs. We are _nowhere near_ that 9
| months on to it. That's all there is to say it.
|
| > If anything, TFA is claiming evidence of an LLM doing
| "most" of what SWEs do "end to end" three months ahead of
| schedule.
|
| TFA based on a model that is so good that it has to be
| kept from us? from the company that literally can't keep
| their app up? From the company who shipped an update that
| didn't launch?
|
| > be my guess, but don't tell falsehoods about the
| evidence you are presenting.
|
| I mean, I literally posted a quote from the CEO of one of
| the two major companies saying that SWEs are 6-12 months
| away from being replaced. This is fantasy talk from a guy
| who is incentivised to have you believe this. If the
| claims are that software is changing, and how we're
| building/deploying software is adapting to that new world
| then yeah that's fair enough. But the current models,
| harnesses and tooling are not replacing an SWE unless
| there's a paradigm shift in the next 3 months. And my
| point is that we appear t be going backwards, not
| forwards.
|
| > didn't make GLP1s fake just because they had the same
| type of hype.
|
| No, GLPs work and that's the difference.
| KronisLV wrote:
| > Every time I mention this
|
| I feel like there's a bunch of factors for why it will
| never be the same for many folks, from the models and
| harnesses, to the domains and existing tests/tooling.
|
| I feel bad for the people for whom it doesn't work, but
| Claude Opus has written most of my code in 2026 so far. I
| had to build some tools around linting entire projects and
| most of my tokens are probably referencing existing stuff
| and parallel review iterations and tests, but it's pretty
| nice and even seeing legacy code doesn't make me want move
| to a farm and grow potatoes.
|
| It might be counter productive to be like: "Oh, just do X!"
| which works for the person suggesting it, and then have to
| do "But have you tried Y?" when it doesn't for the other
| person, if it just keeps being a never ending string of
| what works for one person not working for another.
| maccard wrote:
| > I feel like there's a bunch of factors for why it will
| never be the same for many folks, from the models and
| harnesses, to the domains and existing tests/tooling.
|
| If the argument is "you have to use the right model,
| harness, test and tooling for it to work" then it's not
| replacing software engineers any time soon.
|
| The other thing is - where are all the web apps, mobile
| apps, games, desktop apps, from these 100x productivity
| multipliers. we're 1-2 years into these tools being
| widely mainstream and available and I'm not seeing
| applications that took years to ship before appear at
| 100x the rate, or games being shipped by tiny teams, or
| new ideas of mobile apps coming out at 100x the rate.
| What we do see is vibe coded slop, stability issues with
| massive companies (windows, AWS for example), and mass
| layoffs back to pre-covid levels blamed on AI but
| everyone knows it's a regression to the mean after a
| massive over hiring when money was cheap.
|
| It's like the emperor has no clothes on this topic to me.
| dvfjsdhgfv wrote:
| I wouldn't paint the image in such black terms. LLMs can
| be good in finding bugs and potential issues. And if you
| like, they can be like IntelliSense on steroids. Even
| agentic workflows can be good, e.g. for an initial
| assessment of a new large codebase. And potentially
| millions of other small tasks like writing one-off helper
| scripts etc.
| maccard wrote:
| So which apps are seeing 10x the bug fixes and
| improvements in stability and quality? From my side, I
| see one shot CRUD apps, platforms like AWS and windows
| actively deteriorating, to the point of causing massive
| outages and needing to have development processes changed
| [0]. Who is actually shipping 10x more stuff, or fixing
| 10x more bugs?
|
| [0] https://arstechnica.com/ai/2026/03/after-outages-
| amazon-to-m...
| camdenreslink wrote:
| It actually seems like people are shipping 10x more bugs,
| not fixing 10x more bugs.
| nyrikki wrote:
| The Automation bias issue is something that has been
| raised by many people like myself but mostly ignored. The
| better models get the worse that problem with get, but
| IMHO the implications of the claims are not on the code
| generation side.
|
| The sandwich story in the model card is the bigger issue.
|
| LLMs have always been good at finding a needle in a
| haystack, if not a specific needle, it sounds like they
| are claiming a dramatic increase in that ability.
|
| This will dramatically change how we write and deliver
| software, which has traditionally been based on the idea
| of well behaved non-malfeasant software with a fix as you
| go security model.
|
| While I personally find value in the tools as tools, they
| specifically find a needle and fundamentally cannot find
| all of the needles that are relevant.
|
| We will either have to move to some form of zero trust
| model or dramatically reduce connectivity and move to
| much stronger forms of isolation.
|
| As someone who was trying to document and share a way of
| improving container isolation that was compatible with
| current practices I think I need to readdress that.
|
| VMs are probably a minimum requirement for my use case
| now, and if verified this new model will dramatically
| impact developer productivity due to increased
| constraints.
|
| Due to competing use cases and design choice constraints,
| none of the namespace based solutions will be safe if
| even trusted partners start to use this model.
|
| How this lands in the long run is unclear, perhaps we
| only allow smaller models with less impact on velocity
| and with less essential complexity etc...
|
| But the ITS model of sockets etc.. will probably be dead
| for production instances.
|
| I hope this is marketing or aspirational to be honest. It
| isn't AGI but will still be disruptive if even close to
| reality.
| muggesmuds wrote:
| wasn't there a news story about the app store reviews
| being delayed because of an increase in app influx?
| RugnirViking wrote:
| that doesnt tell us much about the subjective quality of
| the apps in said influx
| fpaf wrote:
| Where are all the apps? It's mostly visible in AI tooling
| itself. Harnesses, vibe coding tools and stuff with
| "claw" in the name saw a cambrian explosion.
|
| And maybe using AI to use AI better is just masturbatory.
| But coders want interesting problems to solve. Pros also
| need software ideas they can monetize. And what problem
| is attracting more investment in money, time and neurons
| than the problem of making AI productive? (I am referring
| only to problems that can be solved in software....)
|
| So the thing with AI is that right now it is both a tool
| AND a potentially very valuable problem to solve, that's
| why most of the AI "productivity" gains go into AI
| itself. At one point this self-refetential phase will
| have to end and people are going to see if these new AI
| tools, harnesses.claw-things are actually applicable to
| things people are willing to pay the real prices for (not
| the subsidized ones).
| JambalayaJimbo wrote:
| I absolutely feel like there has been an explosion of
| software since the release of AI tools. This is a
| subjective assessment anyway...
|
| My company for example has gotten 500% better at creating
| productivity tools.
| gyomu wrote:
| I'm an indie developer and I see the explosion in apps in
| my niche (creative tools for photography/videography).
|
| They wouldn't have taken years to ship before, but easily
| a couple months.
|
| Now the moment any app with any value gets popular, the
| App Store gets flooded with quick vibe coded copycat
| clones (very recognizable AI generated icon included).
|
| The quality is low, but the impact this flood has on the
| market is real.
| laserlight wrote:
| > I feel like there's a bunch of factors for why it will
| never be the same for many folks
|
| Yeah, and the problem arises simply because some people
| are unable to accept the fact. They insist that if LLM-
| assisted coding doesn't work for one, it's because
| "you're holding it wrong".
| dvfjsdhgfv wrote:
| > I had to build some tools around linting entire
| projects
|
| OK, everybody is doing that. And everybody is doing their
| best at making LLMs more reliable when working on non-
| trivial tasks. Yet, it looks like nobody came up with a
| universal solution yet. This is particularly true for
| non-trivial projects.
| mlsu wrote:
| It's because the models response is conditioned on the
| prompt. They are as intelligent as the person using them
|
| In some sense it's a lot like a google search. There's
| this big box of knowledge and you are choosing tokens to
| pluck out of it. The quality of the tokens depends on how
| intelligent you are.
| ofjcihen wrote:
| Don't forget, it also depends on the complexity of the
| work and the experiences of the operator.
|
| The less complex the work and the less experienced the
| operator means more perceived "wow" factor :)
|
| There's definitely an aspect of how you use it though. In
| my work it's mostly been chaining to reduce non-
| determinism.
| GoatInGrey wrote:
| The irony here is that even if one is extracting
| legitimate value from LLMs because they are that much
| smarter than their peers, the process of using LLMs to
| perform all of their skilled labor makes them less
| intelligent.
| je42 wrote:
| Even co-pilot writes most of my code in april 2026.
|
| Further, i don't trust code anymore that hasn't been
| reviewed 3x or more by co-pilot.
|
| If you have asked me 6 months ago I wouldn't have
| expected this change so soon.
| ndr wrote:
| Check out from this onwards and the following point. You
| get a nice summary on top right. Mind that Anthropic alone
| is doing 30B/y annualized already.
|
| Take a snapshot and check again in a few months. It's not
| perfect but it's much more falsifiable than a lot of the
| noise.
|
| https://ai-2027.com/#narrative-2026-04-30
| maccard wrote:
| > Mind that Anthropic alone is doing 30B/Y annualised
| already
|
| How many crypto exchanges were pulling in hundreds of
| millions in funding and doing billions in trades in
| 2021/2022?
|
| That blog post is... really something, I'll give you
| that. Im not entirely sure what else to say about it
| other than that.
| piyh wrote:
| Trade volume and buying API credits are very dissimilar
| ways of measuring value. One can be wash traded into
| oblivion, the other is burning a hole in corporate
| accounts.
| shafyy wrote:
| I'll have some of what you're having
| materialpoint wrote:
| Did you compare AI companies to parents and engineers
| actually delivering value to toddlers? AI companies cannot,
| in any capacity, be regarded as caretakers.
| avaer wrote:
| The parents in this case are profiteering corporations on a
| mission to exploit the child for everything they can get away
| with, almost by definition.
|
| It's a slightly different dynamic.
| bottom999mottob wrote:
| This is literally one the most infantilizing and
| simultaneously insulting analogies I've ever come across on
| this site. Do you really think consumers of the latest AI
| tools have no ability to forecast? The parents in this
| analogy have every incentive to lie
| FridgeSeal wrote:
| I feel like you're muddying 2 different arguments here. Or
| rather, 2 different positions.
|
| You're asserting that people who are tired of this line being
| wheeled out hold a position analogous to "what's the big
| deal, nothing bad happens, just relax". In reality, that's
| only 1 position. The other position is "I understand fully,
| the consequences, but the relentless doomer language is
| tiring in the face of continuing-to-not-eventuate".
| juleiie wrote:
| Don't take it personally but this amount of fear and paranoia
| about death on every corner sounds like a mental illness to
| me. Generalised Anxiety disorder to be precise. Maybe I am
| just not a parent.
|
| In any case there are substances and realiable methods that
| fix whatever paralyzing existential dread anyone struggles
| with daily.
|
| Probably best to use conventional route but I personally use
| special low thc, high cbg weed once a week with a medical
| grade vaporizer and once a year (early autumn) a moderate
| dose of golden teacher mushrooms. Although I understand that
| most people perhaps couldn't due to not managing their own
| business but on a strict employment contract with urine
| tests.
| haritha-j wrote:
| Sure, if the parent's stock price soared if the child dies.
| therealpygon wrote:
| Are you suggesting these researchers somehow have wisdom and
| aren't just guessing, and that everyone else are children too
| naive to understand the technology? It certainly sounds that
| way from the description you are attempting to apply.
|
| This is two parents disagreeing on whether their child will
| automatically grow up to be a psychopath with one parent
| constantly remarking "if you teach that child how to cut
| bread, they will stab everyone later. If you teach that child
| to drive, they will run over everyone later", not the
| "parents know better" situation you describe.
| toraway wrote:
| An analogy that's, quite literally, an appeal to paternalism
| to trust the motivations and pernicious incentive structures
| of the big AI labs.
| jwpapi wrote:
| I agree I can't open any social media no more
| alexey-salmin wrote:
| I think Claude Code with Sonnet 4.6 is already at the level of
| paradigm shift and can change the entire tech industry.
|
| If you're paranoid it doesn't mean you're not being followed.
| If something is overhyped it doesn't mean it's not game-
| changing.
| ofjcihen wrote:
| Oh I agree with you on that. But that's partially why the
| language in the presser falls flat for me.
|
| I mean as an example while web app pen testing I've been
| running and proxying all my traffic through it with
| instructions to find vulnerabilities with instructions
| telling it it's a senior web app security export looking over
| my shoulder. It's already great at that.
|
| Ive even told it to do recon and run pen tests on lists of
| subdomains before (please for the love of god have the right
| harnesses and guardrails before you do this) and woken up to
| paid findings before.
|
| So like I'm in a weird place where this was already happening
| and Mythos is being sold like it wasn't good before?
|
| End ramble :/
| corranh wrote:
| It's great marketing to lead with how the n+1 model is so
| amazing that you can't have it yet.
| Th3Alt3r wrote:
| yeah, they gotta find a way to build hype on every new model
| release
| AlexCoventry wrote:
| Do you think they're lying about the vulnerabilities they claim
| Mythos has found? Seems like a very short-term play, if so.
| nl wrote:
| Well Opus 4.5/4.6 kinda was right?
|
| I mean software development has changed more since then than it
| has in my 30 year software development career.
| _the_inflator wrote:
| I side with you but on the other hand: this is how it works to
| get attention by those who aren't affiliated with computer
| science and AI.
|
| I am totally annoyed as well and put any buzzwords in my
| personal bs filter. Java was revolutionary, the Apple I etc. ;)
|
| On the other hand I see progress! AI enriched press releases
| balance buzzwords and information way better than marketing of
| large companies did before AI.
|
| I remember throwing away an instruction for an electronic
| toothbrush away because - I won't mention the name but have a
| look at the upper tier - instead of putting something like
| "Turn toothbrush on, choose mode by pressing..." it read "Take
| your super awesome premium masterpiece using patented
| technology for the first time in human life now available to
| you by us. Move your finger over to the innovative sensory
| surface, that uses material from rocket scientists and world
| leading designers".
|
| No joke. These were text blocks and repeated - 30 pages for one
| compact one.
|
| The toothbrush is top notch, except for the instructions.
| ofjcihen wrote:
| Hahaha I think we might have the same toothbrush.
|
| That makes sense and I like the analogy.
| raxxorraxor wrote:
| This looks more like another lobby group (quite a bad one) than
| something primarily focused on security.
|
| The "urgency" is very likely mostly appreciated to drive
| policy.
| adam_patarino wrote:
| I've lost trust in anything they say.
|
| The fear marketing is clearly intentional at this point.
| baggachipz wrote:
| And the complicit, click-thirsty tech media falls for it
| every time.
| DonsDiscountGas wrote:
| Everybody remembers the fable of the boy who cried wolf and how
| he died at the end. Left out of the story is the multiple other
| villagers who died of starvation because their flock of sheep
| was eaten. So because they didn't want to feel like suckers.
| Tuning out completely because of the existence of false
| positives is not a good choice.
| heliumtera wrote:
| At launch, a technology is considered dangerous for being too
| powerful.
|
| 3 months later, you are an absolute idiot to still be using
| that useless model. Are you not using glasswing 2-01 high? Oh,
| yeah, glasswing from 3 months ago is absolutely worthless,
| every viber knows, it's your fault for holding it wrong.
|
| For once you should not get too excited for new models release
| and words and adjectives promising things. Honestly it's your
| fault humanity lost its humanity and we just have words words
| words and mass schizophrenia
| dkersten wrote:
| And every single time what they release is underwhelming.
|
| Remember how Sam spent like a year talking about how scary
| close GPT-5 was to AGI and then when it did finally come out...
| it was kinda meh.
| FiberBundle wrote:
| To me it makes absolutely zero sense that they would decide to
| not release the model to the public because of the effects that
| it would have due to its exploitation capabilities. Previous
| models were also capable of providing harmful information, yet
| that wasn't a problem, because models can actually be
| effectively censored using RHLF. So what is preventing
| Anthropic to simply forbid the model from letting people vibe-
| code exploits???
| gchadwick wrote:
| Remember OpenAI decided GPT 2 was far too dangerous to unleash
| upon the world when they first trained it!
| nearbuy wrote:
| That's an editorialized headline. What they actually wrote
| was that it could be used to "generate misleading news
| articles, impersonate others online, automate the production
| of abusive or faked content to post on social media, [or]
| automate the production of spam/phishing content" and that
| they are aware other researchers have the ability to
| reproduce and open source their results, but this would give
| the community some time to decide how to proceed.
|
| They were correct.
| davenporten wrote:
| I came across this article just this morning saying AMD
| researchers, who hitherto have relied on Claude Code heavily,
| have noticed degraded performance in the recent update:
| https://www.theregister.com/2026/04/06/anthropic_claude_code...
|
| Claude Code and Glasswing are not the same, but presumably they
| have a lot of overlap under the hood. I feel like while AI is
| certainly advancing in major ways, there will always be the up
| and down of new software releases.
| throwawayq3423 wrote:
| > I can't be the only person who's getting tired of hearing
| about how every new iteration is going to spell doom/be a
| paradigm shift/change the entire tech industry etc.
|
| There's a little bit of a grading your own homework aspect to
| companies being able to declare their new models revolutionary.
|
| It doesn't mean they're wrong, but there is a clear conflict of
| interest.
| akmiller wrote:
| Hasn't almost every model created a paradigm shift lately?
| Maybe it's you who has moved the needle on what a paradigm
| shift means?
| aagha wrote:
| https://news.ycombinator.com/item?id=47682262
| eranation wrote:
| It feels to me full with marketing in the guise of trying to
| save the world from their own making. "we have a model so
| strong we can't release it, here are all the details of why
| it's so good, but don't ask for access, you can't get it, it's
| too risky for your own good"
|
| Something smells really really weird:
|
| 1. Per the blog post[0]: "This was the most critical
| vulnerability we discovered in OpenBSD with Mythos Preview
| after a thousand runs through our scaffold. Across a thousand
| runs through our scaffold, the total cost was under $20,000 and
| found several dozen more findings"
|
| Since they said it was patched, I tried to find the CVE, it
| looks like Mythos indeed found a 27 years old OpenBSD bug
| (fantastic), but it didn't get a CVE and OpenBSD patched it and
| marked it as a reliability fix, am I missing something? [1]
|
| 2. From the same post, Anthropic red team decided to do a
| preview of their future responsible disclosure (is this a
| common practice?): "As we discuss below, we're limited in what
| we can report here. Over 99% of the vulnerabilities we've found
| have not yet been patched" [0] So this is great, can't wait to
| see the actual CVEs, exploitability, likelihood, peer review,
| reproducibility, the kind of things the appsec community has
| been doing for at least the last 27 years since the CVE concept
| was introduced [2]
|
| 3. On the same day, an actual responsible disclosure, actual
| RCEs, actual CVEs, in Claude Code, that got discovered mostly
| because of the source code leak, I don't see anyone talking
| about it (you probably should upgrade your Claude Code though).
|
| CVE-2026-35020 [3] CVE-2026-35021 [4] CVE-2026-35022 [5]
|
| Do with this information as you may...
|
| [0] https://red.anthropic.com/2026/mythos-preview/
|
| [1] https://www.openbsd.org/errata78.html (look for 025)
|
| [2] https://www.cve.org/Resources/General/Towards-a-Common-
| Enume...
|
| [3] https://www.cve.org/CVERecord?id=CVE-2026-35020
|
| [4] https://www.cve.org/CVERecord?id=CVE-2026-35021
|
| [5] https://www.cve.org/CVERecord?id=CVE-2026-35022
| fullstackchris wrote:
| Agreed. Do we have any information on what these
| "vulnerabilities" actually are? Every vulnerability is
| typically immediately reported to CVE or NIST... are these "so
| destructive" they have to be kept behind closed doors? Give me
| a break...
| Ms-J wrote:
| Anthropic and ClosedAI are some of the biggest bullshitters in
| the industry.
|
| The is no moat, no special "capability" and when the time comes
| when we can run these models on our own, they will be cheap SaaS
| gimmicks marketed to corporate and making more slop pictures for
| social media.
| 5d41402abc4b wrote:
| Are there any local models that i can setup to run on my code as
| part of CI?
| rossjudson wrote:
| Security by obscurity is over. The security vs usability balance
| is about to get a hard reset.
|
| I think a number of black swan events are imminent, and it will
| substantially change the financial calculus that decides to put
| security behind revenue.
|
| Any hole will be found, and any hole will be exploited. Plug as
| many holes as you can, and make lateral movement as painful as
| possible.
| sam0x17 wrote:
| It's all just really genius marketing. In 6 months Mythos will be
| nothing special, but right now everyone is being manipulated into
| fearing its release, as a marketing ploy.
|
| This is the same reason AI founders perennially worry in public
| that they have created AGI...
| sam0x17 wrote:
| It's effectively 2026's version of "Doctors hate this one weird
| trick!"
| declan_roberts wrote:
| The fact that they are not going to release this DANGEROUS
| model is also a huge tell that it's nothing but an
| incremental improvement over the status quo.
| declan_roberts wrote:
| I can't believe the effectiveness of this type of marketing.
| It's one-shotting normie journalist and getting a lot of press
| for what is ultimately going to turn out to be an incrementally
| improved model.
|
| I'm sure all they've done here is spend unlimited tokens to
| find bugs in mostly open source projects (and fuzz some closed
| source ones).
| sensanaty wrote:
| I'm so tired of the astroturfing from Anthropic literally
| everywhere. Every single forum, every single thread anywhere on
| the internet is filled with their bots muddying up the
| conversation, it's so tiring.
| MattRix wrote:
| Where is the astroturfing? From what I can tell it's maybe
| the fastest growing product/company ever. Anthropic's
| products have completely changed how software development
| gets done across the entire industry, especially in the past
| four months. The level of hype seems entirely justified to me
| (and I say this as an OpenAI Codex user).
| stratos123 wrote:
| I find it very unlikely that Mythos will "be nothing special".
| Current Opus is already "special" enough to find dozens of real
| bugs in Firefox and the Linux kernel, and Mythos is, it seems,
| a full OOM above it.
| nicce wrote:
| Most flagship models have found real bugs. Not sure if Opus
| deserves mention alone. Even open models have found many.
| sam0x17 wrote:
| What I mean by "nothing special" is 1 year from now you will
| say it is an extremely limited model compared to whatever is
| out then
| MattRix wrote:
| This is what happens when you have growth that is
| exponential instead of linear!
| lifeisstillgood wrote:
| Nicolas Carlini talks about it here on Security, Cryptography,
| Whatever podcast -
| https://podcasts.apple.com/gb/podcast/security-cryptography-...
| VadimPR wrote:
| I'm not one to believe the Silicon Valley hype usually (GPT-2
| being too dangerous to release, AI giving us UBI, and so on), but
| having run Claude Opus 4.6 against my codebase (a MUD client)
| over the weekend, I can believe this assessment.
|
| Opus alone did a good job of identifying security issues in my
| software, as it did with Firefox [1] and Linux [2]. A next-
| generation frontier model being able to find even more issues
| sounds believable.
|
| That said, this is script kiddies vs sql injections all over
| again. Everyone will need to get their basic security up on the
| new level and it will become the new normal. And, given how
| intelligence agencies are sitting on a ton of zero-days already,
| this will actually help the general public by levelling out the
| playing field once again.
|
| 1 - https://www.anthropic.com/news/mozilla-firefox-security 2 -
| https://neuronad.com/ai-news/claude-code-unearthed-a-23-year...
| attentive wrote:
| Is there timeline mentioned anywhere on when any of this will be
| available for unprivileged public as in soon, not soon, never?
| Surac wrote:
| namedropping hell.
| finchisko wrote:
| Wait, isn't it how Skynet started?
| jiusanzhou wrote:
| The $100M in credits for open-source scanning is the most
| interesting part here. The real bottleneck was never finding
| vulns in high-profile projects -- it was the long tail of
| critical dependencies maintained by one or two people who don't
| have time or resources for serious auditing. If Glasswing
| actually reaches those maintainers, it could meaningfully reduce
| the attack surface that supply chain attacks exploit.
| jusling wrote:
| so it looks like ai-slop replies have made their way to HN...
| peyton wrote:
| Unfortunate. I'm so sick of hearing what things are not, or
| what's real, or what's interesting.
| m132 wrote:
| I must say the combo of an em-dash stuck right in the middle of
| "it was never X, it was Y" made me chuckle
| lasky wrote:
| The hype machine is alive and well in silicon valley.
| steinwinde wrote:
| From a non-US perspective this must be disquieting to read: Not
| so much that Anthropic considers only US companies as partners.
| But what does Anthropic do to prevent malicious use of its
| software by its own government?
|
| > Anthropic has also been in ongoing discussions with US
| government officials about Claude Mythos Preview and its
| offensive and defensive cyber capabilities. As we noted above,
| securing critical infrastructure is a top national security
| priority for democratic countries--the emergence of these cyber
| capabilities is another reason why the US and its allies must
| maintain a decisive lead in AI technology.
|
| Not a single word of caution regarding possible abuse. Instead
| apparent support for its "offensive" capabilities.
| alexey-salmin wrote:
| In my view it would be extremely strange if it was any other
| way round. Anthropic is the US based company. There are no
| "citizens of world" at that scale, or at almost any other scale
| for that matter.
| khafra wrote:
| > what does Anthropic do to prevent malicious use of its
| software by its own government?
|
| Anthropic has ameliorated that danger by being designated a
| supply-chain risk by the DoW, preventing the USG from using it.
| saretup wrote:
| Even more 'disquieting' when you take into account who's
| currently the president of US.
|
| "A whole civilization will die tonight, never to be brought
| back again. I don't want that to happen, but it probably will."
| - Donald Trump
| aurareturn wrote:
| When I was reading https://ai-2027.com, which is quite a
| scary read, I couldn't help but think the US president being
| mentioned in the story acts too rational compared to the real
| world. It can get a lot crazier than this fictional piece.
| Rover222 wrote:
| The art of the deal, baby
| dyauspitr wrote:
| Yeah this is bad timing for frontier progress.
| tomaskafka wrote:
| There is very little Anthropic can do - that job is up to US
| citizens creating and enforcing checks and balances. You can't
| ask a company legally bound by your country laws (made by your
| own representatives) to protect you or anyone else from said
| laws. That is your job.
|
| And it is other countries job to protect themselves from other
| countries weapons. As EU citizen I'd much rather if EU had a
| frontier model on par, but here we are.
| steinwinde wrote:
| According to Dario Amodei, companies bear a lot of
| responsibility and must act on this. Just read
| https://www.darioamodei.com/essay/the-adolescence-of-
| technol... . But it seems that he has given up on this, even
| if he has a president that demands "complete and total
| control of Greenland" etc. What "allies" is this Anthropic
| statement referring to anyway?
| signatoremo wrote:
| Anthropic stood up to the Pentagon because they were worried of
| potential abuse of their model. Never before a US company was
| labeled supply chain risk by the US government. That's a lot of
| business. Action speaks louder than words.
|
| As for what your country can do, it's up to you to decide,
| isn't it? Instead of complaining about the US, think about the
| alternatives. Do you trust China to be your partner? Suppose
| you are being objective and say no, then what do your country
| need to do?
|
| You have to decide whether AI capability is critical that your
| country must own. What factors prevent it from happening in the
| first place, what need to change and whether you accept changes
| that may come as the results.
|
| On the other hand, if you say that AI is just a bubble, that
| the huge investment pouring into it is just greed and fraud,
| then I suppose you are ok with the status quo.
| asG11 wrote:
| "We have also extended access to a group of over 40 additional
| organizations that build or maintain critical software
| infrastructure so they can use the model to scan and secure both
| first-party and open-source systems."
|
| Yeah, yeah. Back in the day IBM Purify gave access to software
| organizations and found very little. Of course they did not have
| the free money of a marketing driven organization run by a weirdo
| (Amodei) that got rich by stealing and laundering IP.
|
| This will fizzle out and the weirdo will have to pivot to their
| next marketing scheme.
| ahmaman wrote:
| Moving forward, wonder if such AI capabilities would widen the
| security gap between open-source software vs. proprietary?
| 123malware321 wrote:
| I don't know anyone reviewing these tools that is impressed who
| is also someone who earns they paycheck doing bugbounties and
| finding actual CVE.
|
| Generally these things only find memory corruption stuff which is
| almost never the type of bug you're looking for, and it costs a
| lot which negates your bug bounty payout.
|
| Each time they preach, ooh, 0day found, bla bla.
|
| In this domain you need to be specific or you are just yelling
| clickbait into the wind.
|
| What type of 0day, what did the exploit actually look like.
|
| 'complex 4 stage with heap spray' - that sounds really simple
| actually.... complex for memory corruption goes into multi-
| process, maybe things between kernel/usermode, or crazy 18-20
| stage exploits people pop against things like MS Teams etc....
|
| Even if there were some cool results by any of these projects,
| the amount of nonsense blurted out in articles around them really
| makes them seem useless tools that are overmarketed by a bunch of
| excited children who dont really know what they are doing.
|
| Get a dopamine hit, post on reddit, LOL. Hacking the planet
| (powered by Claude -_-)
| sensanaty wrote:
| You'd think with this "terrifying" powerful model of theirs they
| could have a few less red bars on their status page[1], but
| apparently the hyper-intelligence is only capable of pulling off
| uber-sophisticated cyber attacks and not making a frontend that
| doesn't shit itself constantly, curious.
|
| [1] https://status.claude.com/
| abhikjain360 wrote:
| One argument can be made that this is an issue of there simply
| not being enough compute in the world to meet the demand for
| claude's LLMs right now, and not really an issue with their
| infra setup or architecture.
| yalogin wrote:
| Has anyone played with the released versions of Claude and tried
| to create exploits? I cannot imagine it not being able to craft
| one if guided, unless the tooling around it doesn't allow it
| punnerud wrote:
| Simon Willis (guy behind Django) told about this 5days ago (19min
| in): https://youtu.be/wc8FBhQtdsA?si=OeA5qzbWGqDY8Vu4
| solid_fuel wrote:
| This is the same company that accidentally released the source
| for one of their flagship products last week and has been
| furiously DMCA-ing every repository that even mentions claude in
| the days since.
| thewhitetulip wrote:
| This is also the same company who uses electron for their
| tooling rather than platform specific binaries generated by
| Opus! If their LLMs are that good why do they need to use
| electron?
| bdeol22 wrote:
| The uncomfortable bit isn't tooling--it's cadence. When the
| threat model shifts faster than your review loop can honestly re-
| run, you don't get security, you get paperwork that pretends
| nothing changed.
| willamhou wrote:
| One thing I keep thinking about with AI security is that most of
| the focus is on model behavior -- alignment, jailbreaks,
| guardrails. But once agents start calling tools, the attack
| surface shifts to the execution boundary. A request can be
| replayed, tampered with, or sent to the wrong target, and the
| server often has no way to distinguish that from a legitimate
| call.
|
| Cryptographic attestation at the tool-call level (sign the
| request, verify before execution) would close a gap that
| behavioral controls alone can't cover. Curious whether
| Glasswing's threat model includes the agent-to-tool boundary or
| focuses primarily on the model layer.
| cmiles8 wrote:
| I'm sure it's a decent model. But it's also clear folks are
| running out of runway and desperate to find something that sticks
| and keeps the party going.
|
| All the promises of amazing things in general work never
| happened. Companies consistently say they're seeing no ROI. The
| AI crowd now hard pivots to cyber and, right out of the Palantir
| playbook, runs with the "our stuff is so amazing we can't talk
| about it, but trust us bro" move that isn't really fooling
| anyone.
|
| Meanwhile the folks let in on the "secret" are those that also
| desperately need for the hype to continue to protect their own
| positions in this game.
|
| Look forward to a model upgrade but the hype fluff games are
| getting old. Watching OpenAI completely crash out of pole
| position on the hype train though has been at least amusing.
| Rover222 wrote:
| With Anthropic able to use this model internally (since
| February), is this the kickoff of ramping up the flywheel of
| recursive self improvement of AI? It seems like as long as there
| are still humans in the loop at most steps, exponential recursion
| isn't possible.
| cmiles8 wrote:
| So we're meant to believe that Anthropic is sitting on a world
| ending cyber tool that writes God-like code while just forgetting
| that a week ago the same company leaked its source code on the
| internet and was ribbed for how shit it was.
|
| Got it.
| burntcaramel wrote:
| Previously Anthropic subscribers got access to the latest AI but
| it seems like there's a League of Software forming who have
| special privileges. To make or maintain critical software will
| you have to be inside the circle?
|
| Who gates access to the circle? Anthropic or existing circle
| members or some other governance? If you are outside the circle
| will you be certain to die from software diseases?
|
| Having been impressed by LLMs but not believing the AGI hype, I
| now see how having access to an information generator could be so
| powerful. With the right information you can hack other
| information systems. Without access to the best information you
| may not be able to protect your own system.
|
| I think we have found the moat for AI. The question is are you
| inside or outside the castle walls?
| thatxliner wrote:
| I feel like people keep forgetting that it's possible to code
| without ai, but yes arguably a lot slower, typically.
| kouru225 wrote:
| They've been trying their hardest to find a moat for 5 years,
| and nothing seems to stick. At first it seemed like access to
| the model could be a moat but then llama and deepseek came out.
| Then it seemed like the hardware requirements could be a moat
| but small local AI just kept getting more efficient. Now
| they're trying to gate keep access to the models again under
| the guise of security, but we probably got like t minus 2 weeks
| before an equivalent model is released by someone
|
| American AI desperately wants AI to intensify the wealth
| disparity and therefore justify the wealth grab that the rich
| have done for the last 3 decades and AI is just not cooperating
| 6thbit wrote:
| It's only a moat if you believe no competing lab will achieve
| similar or better results in a large enough time frame to
| profit from it.
| theptip wrote:
| Thats not what a moat means in business. It doesn't mean
| impregnable, it just means expensive or difficult to cross.
|
| It is absolutely a moat if only $1T companies can afford the
| capex to compete.
| 6thbit wrote:
| non-zero already existing companies can afford to compete,
| so its not a great preliminary line of defense against your
| main enemies.
|
| If the existing competition's turn around time to out-
| benchmark you is in the order of weeks, its an absolutely
| terrible moat.
| rubises wrote:
| The harder problem isn't finding vulnerabilities -- it's
| preventing AI from violating constraints in the first place.
| Prompt-level safety is probabilistic. Filesystem-level
| constraints (mkdir Jin /behavior) are deterministic. The AI can't
| violate a rule that's physically encoded as a folder path in its
| system prompt.
| throwaway911282 wrote:
| Pumping is taken to a new level.. the model is God like that it
| can't be released as it is.. this must be a joke.
| asdewqqwer wrote:
| There is a huge gap between the shining examples and actual use
| case: What is the false positive rate? How to judge false
| positive?
|
| If you need 1000 run that cost 20000 USD to find a vulnerability,
| and you need 2000 USD to generate a exploit (which makes it self-
| verifiable to be not false positive), than your cost is not 22000
| USD but 1000x2000+2000 which is 2 million USD: you have to try
| generating exploit for every trial before you know it is true, or
| you need to hire one (or several) senior security people to audit
| every single of them.
|
| A broken clock being correct twice a day is not impressive.
| sunk1st wrote:
| My impression from the article is that it took $20,000 to
| perform all 1,000 runs.
| asdewqqwer wrote:
| yet the poc exploit itself take $2000 and one day, I don't
| know how the math works, maybe there is some extremely clever
| way to figure out runs that are not worthy to attempt
| exploit.
| cerved wrote:
| Anthropic should run it on their own code
| DigitalArchivst wrote:
| Do folks recommend that family and friends ensure their systems
| are updated, and that they are using Bitwarden or 1Password? Or
| is that alarmist?
| aurizon wrote:
| This has all happened before, back in the day we has spinners and
| weavers, then we got the spinning Jenny(Engine) and this made
| thread so cheap we needed to speed up weaving = machine
| weavers(AKA automatic looms) and we had people who hated
| them.https://en.wikipedia.org/wiki/Luddite We all know how that
| ended up. We have an analogous hand task = coding versus coding
| machines. They will probably eliminate 80-95% of coding, as the
| spinners/weavers went away, but there remains a residual
| artisanal spinner/weaver industry that carries on at a lower
| pace. In a similar way this machine code will have the coupled
| ability to make some code and then test it in use with it's own
| AI in a repeated/recursive way to make/test/improve code at a
| rate 10,000 to 1 million times faster than a human. Each module
| can then be tested in millions of interactively monitired ways to
| find/fix/kill bad modules. It can also pentest in a similar
| manner, assaulting a system with a blizzard of attack/reset hits
| to find any bugs etc. Each assault that works might use a human
| or AI to trouble shoot. This is like the old armored night, once
| he was unhorsed the peasants would have at him with needles at
| his his joints/eyes unless his fellows save him = gone. So this
| might well reduce low end jobs, but they will still need high end
| coders to eliminate all flaws in the armor of your code. I might
| be simplistic, but I see a parallel in sub 5 nm chip design where
| the design machines have eliminated almost all of the old hand
| work.
| User23 wrote:
| How much of Mythos's internals will researchers be able to
| recover from the flood of patches?
| tombelieber wrote:
| I think this new model will empower everyone in the world to have
| higher quality of software, more secure software. not less
| chenzhekl wrote:
| It feels like the current trend is a bit scary: the more AI
| advances, the more people with money and resources will gain
| disproportionately greater advantages. For example, they can make
| their own software more secure, while also finding it easier to
| discover ways to attack other software.
| IMTDb wrote:
| You can already do that today by hiring a security researcher.
| I can guarantee you that Apple has access to people of a higher
| caliber than my startup.
|
| I could see a world where 1 year from now I can have glassing
| do a full sweep of my codebase for a given price (say: $10k).
| Running that once a year is within my means and would make my
| software much more secure than it is today.
| powvans wrote:
| I spend well over that of my employers money on pentesting
| every year. I'm absolutely certain Claude could perform as
| good or better a job using what's available today.
|
| It had crossed my mind that an AI agent pentester would be an
| interesting product to build. Once again though, the labs are
| just going to build it because it's a thin thin wrapper.
|
| Beyond existing software with vulnerabilities, the really
| important aspect of this for Anthropic et al is that the
| gigatons of code that are being generated every day needs to
| be secured.
| StrauXX wrote:
| There are quite a few such startups already out there.
| Results are mixed so far. Though I believe they get much
| better over the coming months and years.
| hhh wrote:
| AWS has one as a managed service.
| dyauspitr wrote:
| Yeah but even Carlini who is a good security researcher said
| he has found more valid vulnerabilities in the last week than
| his entire career before this. That sounds like it's clearly
| better/faster/cheaper than a human security researcher that
| would cost $300,000 a year.
| willy_k wrote:
| Sounds normal to me!
|
| i.e. it may be a step change and that could very well have
| distinct and noticeable real world effects, like other
| technologies have in the past, but it's nothing fundamentally
| new.
| btucker wrote:
| This has increasingly been my take. If we accept that AI is an
| amplifier of impact, then it follows it will amplify
| disparities.
| jaspanglia wrote:
| what they will eventually do is, deliberately have more control
| what people wants and working for. We don't trust such
| institutions after witnessing GATES thuggery all over.
| eranation wrote:
| Few thoughts
|
| 1. Per the blog post[0]: "This was the most critical
| vulnerability we discovered in OpenBSD with Mythos Preview after
| a thousand runs through our scaffold. Across a thousand runs
| through our scaffold, the total cost was under $20,000 and found
| several dozen more findings"
|
| Since they said it was patched, I tried to find the CVE, it looks
| like Mythos indeed found a 27 years old OpenBSD bug (fantastic),
| but it didn't get a CVE and OpenBSD patched it and marked it as a
| reliability fix, am I missing something? [1]
|
| 2. From the same post, Anthropic red team decided to do a preview
| of their future responsible disclosure (is this a common
| practice?): "As we discuss below, we're limited in what we can
| report here. Over 99% of the vulnerabilities we've found have not
| yet been patched" [0] So this is great, can't wait to see the
| actual CVEs, exploitability, likelihood, peer review,
| reproducibility, the kind of things the appsec community has been
| doing for at least the last 27 years since the CVE concept was
| introduced [2]
|
| 3. On the same day, an actual responsible disclosure, actual
| RCEs, actual CVEs, in Claude Code, that got discovered mostly
| because of the source code leak, I don't see anyone talking about
| it (you probably should upgrade your Claude Code though).
|
| CVE-2026-35020 [3] CVE-2026-35021 [4] CVE-2026-35022 [5]
|
| Not making any opinion, just thought it's worth sharing, for some
| perspective.
|
| [0] https://red.anthropic.com/2026/mythos-preview/
|
| [1] https://www.openbsd.org/errata78.html (look for 025)
|
| [2] https://www.cve.org/Resources/General/Towards-a-Common-
| Enume...
|
| [3] https://www.cve.org/CVERecord?id=CVE-2026-35020
|
| [4] https://www.cve.org/CVERecord?id=CVE-2026-35021
|
| [5] https://www.cve.org/CVERecord?id=CVE-2026-35022
|
| Edit: if it was not obvious, these CVEs on Claude Code were found
| by an independent security researcher (Phoenix security) and not
| by Anthropic / Mythos.
| 6thbit wrote:
| Now we have to wonder if they ran Mythos on their Calude source
| and it missed it or why they chose not to run it.
|
| I do agree and wonder why that's not marked as security. In
| their security page [0] it says: > Since exploitability is not
| proven for many of the fixes we make, do not expect the
| relevant commit message to say "SECURITY FIX!".
|
| Does that mean they considered it not to be exploitable?
|
| [0] https://www.openbsd.org/security.html
| eranation wrote:
| I really don't know, all I know is that usually when you find
| a critical vulnerability, and it's patched, it comes with a
| CVE, even a low one, that's the process for the past 27 years
| when the CVE program started (as old as the vulnerability
| itself it seems..) but maybe with AI-native, CVEs don't
| matter because everyone will just rewrite their clean room
| open source alternative (I wish this was a joke...)
| mlvvkviz wrote:
| they built a model so powerful they won't release it. but they
| couldn't secure claude code from a source code leak. the model is
| so advanced they're paying $100M to get big tech to adopt it. the
| launch video reads like verified amazon reviews. the gap between
| the narrative and the reality is the whole story here.
| waffletower wrote:
| My comment is a completely unsubstantiated conspiracy theory: the
| choice of model name, Mythos, seems out of character for
| Anthropic models, and one can easily wonder if the model truly
| exists as the name suggests. It could instead be a symbolic model
| used by colluding companies (and perhaps even governments) to
| establish a reference limit upon what models will be publicly
| accessible, period. Probably a terrible theory as it could spell
| doom for frontier model developing companies' business models --
| setting the bar already would likely commodify LLMs via open
| source models quite quickly. But the name "Mythos" is such a
| strange choice for this model and the circumstances surrounding
| its release.
___________________________________________________________________
(page generated 2026-04-08 23:01 UTC)