[HN Gopher] Project Glasswing: Securing critical software for th...
___________________________________________________________________
Project Glasswing: Securing critical software for the AI era
Related: _Assessing Claude Mythos Preview 's cybersecurity
capabilities_ - https://news.ycombinator.com/item?id=47679155
_System Card: Claude Mythos Preview [pdf]_ -
https://news.ycombinator.com/item?id=47679258 Also: _Anthropic 's
Project Glasswing sounds necessary to me_ -
https://news.ycombinator.com/item?id=47681241
Author : Ryan5453
Score : 728 points
Date : 2026-04-07 18:09 UTC (4 hours ago)
(HTM) web link (www.anthropic.com)
(TXT) w3m dump (www.anthropic.com)
| Ryan5453 wrote:
| Pricing for Mythos Preview is $25/$125, so cheaper than GPT 4.5
| ($75/$150) and GPT 5.4 Pro ($30/$180)
| cassianoleal wrote:
| Where did you get that from?
|
| From TFA:
|
| > We do not plan to make Claude Mythos Preview generally
| available
| Tiberium wrote:
| From the article:
|
| > Anthropic's commitment of $100M in model usage credits to
| Project Glasswing and additional participants will cover
| substantial usage throughout this research preview.
| Afterward, Claude Mythos Preview will be available to
| participants at $25/$125 per million input/output tokens
| (participants can access the model on the Claude API, Amazon
| Bedrock, Google Cloud's Vertex AI, and Microsoft Foundry).
| underdeserver wrote:
| Key point: available _to participants_.
| conradkay wrote:
| permanent underclass has arrived :(
| conradkay wrote:
| For comparison, 5x the cost of Opus 4.6, and 1.67x for Opus 4.1
|
| I think this would be _very_ heavily used if they released it,
| completely unlike GPT 4.5
| adi_kurian wrote:
| Opus 4 & 4.1 are still on Vertex+Bedrock @ $75/1mm out. They
| were used very heavily and in my subjective opinion are
| better than 4.5 and 4.6.
| breakingcups wrote:
| Interesting, what makes them better to you?
| LoganDark wrote:
| It's nice to know that they continue to be committed to
| advertising how safe and ethical they are.
| rvz wrote:
| They are not our friends and _are_ the exact opposite of what
| they are preaching to be.
|
| Let alone their CEO scare mongering and actively attempting to
| get the government to ban local AI models running on your
| machine.
| simianwords wrote:
| How would you expect them to behave if they were your
| friends?
| ethin wrote:
| IMO (not the GP) but if Anthropic were my friends I would
| expect them to publish research that didn't just inflate
| the company itself and that was both reproduceable and
| verifiable. Not just puff pieces that describe how ethical
| they are. After all, if a company has to remind you in
| every PR piece that they are ethical and safety-focused,
| there is a decent probability that they are the exact
| opposite.
| Miraste wrote:
| They are a for-profit company, working on a project to
| eliminate all human labor and take the gains for
| themselves, with no plan to allow for the survival of
| anyone who works for a living. They're definitionally not
| your friends. While they remain for-profit, their specific
| behaviors don't really matter.
| simianwords wrote:
| I work for a tech company that eliminates a form of human
| labour and they remain for profit
| Miraste wrote:
| Sure, most tech companies eliminate some form of human
| labor. Anthropic aims to eliminate _all_ human labor,
| which is very different.
| SilverElfin wrote:
| I agree attempting to ban local AI models or censor them, is
| not appropriate. At the same time, they do seem far more
| ethical and less dangerous than other AI companies. And I
| include big tech in that - a bunch of greedy companies that
| just want to abuse their monopoli ... I mean moats.
| raldi wrote:
| In what ways is Anthropic different from a hypothetical
| frontier lab that you would characterize as legitimately safe
| and ethical?
| LoganDark wrote:
| I'm just a little frustrated they keep going on about how
| safe and ethical they are for keeping the more advanced
| capabilities from us. I wish they would wait to make an
| announcement until they have something to show, rather than
| this constant almost gloating.
| 0x3f wrote:
| Its existence is possible.
| ehutch79 wrote:
| Just include 'make it secure' in the prompt. Duh.
|
| /s
| zachperkel wrote:
| Mythos Preview has already found thousands of high-severity
| vulnerabilities, including some in every major operating system
| and web browser.
|
| Scary but also cool
| fsflover wrote:
| Every piece of software definitely has serious vulnerabilities,
| perfection is not achievable. Fortunately we have another
| approach to security: security through compartmentalization.
| See: https://qubes-os.org
| syndeo wrote:
| Once you get the compartmentalization working well, and "all"
| of the vulnerabilities are out of it too, of course...
|
| But even then you'll have users putting things in the same
| compartment for convenience, rather than leaving them
| properly sequestered.
| dakolli wrote:
| Or more likely, its just an exaggeration or lie.
| jryio wrote:
| Let's fast forward the clock. Does software security converge on
| a world with fewer vulnerabilities or more? I'm not sure it
| converges equally in all places.
|
| My understanding is that the pre-AI distribution of software
| quality (and vulnerabilities) will be massively exaggerated. More
| small vulnerable projects and fewer large vulnerable ones.
|
| It seems that large technology and infrastructure companies will
| be able to defend themselves by preempting token expenditure to
| catch vulnerabilities while the rest of the market is left with a
| "large token spend or get hacked" dilemma.
| mlinsey wrote:
| I'm pretty optimistic that not only does this clean up a lot of
| vulns in old code, but applying this level of scrutiny becomes
| a mandatory part of the vibecoding-toolchain.
|
| The biggest issue is legacy systems that are difficult to patch
| in practice.
| pipo234 wrote:
| Wait. Wasn't AI supposed to alleviate the burden of legacy
| code?!
| mlinsey wrote:
| If we have the source and it's easy to test, validate, and
| deploy an update - AI should make those easier to update.
|
| I am thinking of situations where one of those aren't true
| - where testing a proposed update is expensive or
| complicated, that are in systems that are hard to
| physically push updates to (think embedded systems) etc
| rattlesnakedave wrote:
| Legacy code, not the running systems powered by legacy code
| buzzerbetrayed wrote:
| If you're still an AI skeptic at this point, I don't know
| what sort of advancement could convince you that this is
| happening.
| wslh wrote:
| I imagine that some levels of patching would be improving as
| well, even as a separate endeavor. This is not to say that
| legacy systems could be completely rewritten.
| qingcharles wrote:
| I could see some of these corps now being able to issue more
| patches for old versions of software if they don't have to
| redirect their key devs onto prior code (which devs hate). As
| you say though, in practice it is hard to get those patches
| onto older devices.
|
| I'm looking at you, Android phone makers with 18 months of
| updates.
| phist_mcgee wrote:
| Yeah but who pays the enormous cost?
| timschmidt wrote:
| Most vulnerabilities seem to be in C/C++ code, or web things
| like XSS, unsanitized input, leaky APIs, etc.
|
| Perhaps a chunk of that token spend will be porting legacy
| codebases to memory safe languages. And fewer tokens will be
| required to maintain the improved security.
| torginus wrote:
| I think most vulnerabilities are in crappy enterprise
| software. TOCTOU stuff in the crappy microservice cloud app
| handling patient records at your hospital, shitty auth at a
| webshop, that sort of stuff.
|
| A lot of these stuff is vulnerable by design - customer
| wanted a feature, but engineering couldnt make it work
| securely with the current architecture - so they opened a
| tiny hole here and there, hopefully nobody will notice it,
| and everyone went home when the clock struck 5.
|
| I'm sure most of us know about these kinds of vulnerabilities
| (and the culture that produces them).
|
| Before LLMs, people needed to invest time and effort into
| hacking these. But now, you can just build an automated vuln
| scanner and scan half the internet provided you have enough
| compute.
|
| I think there will be major SHTF situations coming from this.
| timschmidt wrote:
| Yeah. Crufty cobbled together enterprise stuff will suffer
| some of the worst. But this will be a great opportunity for
| the enterprise software services economy! lol.
|
| I honestly see some sort of automated whole codebase
| auditing and refactoring being the next big milestone along
| the chatbot -> claude code / codex / aider -> multi-agent
| frameworks line of development. If one of the big AI corps
| cracks that problem then all this goes away with the click
| of a button and exchange of some silver.
| pants2 wrote:
| Software security heavily favors the defenders (ex. it's much
| easier to encrypt a file than break the encryption). Thus with
| better tools and ample time to reach steady-state, we would
| expect software to become more secure.
| justincormack wrote:
| Software security heavily favours the attacker (ex. its much
| easier to find a single vulnerability than to patch every
| vulnerability). Thus with better tools and ample time to
| reach steady-state, we would expect software to remain
| insecure.
| fsflover wrote:
| This is only true if your approach is security through
| correctness. This never works in practice. Try security
| through compartmentalization. Qubes OS provides it
| reasonably well.
| pants2 wrote:
| If we think in the context of LLMs, why is it easier to
| find a single vulnerability than to patch every
| vulnerability? If the defender and the attacker are using
| the same LLM, the defender will run "find a critical
| vulnerability in my software" until it comes up empty and
| then the attacker will find nothing.
|
| Defenders are favored here too, especially for closed-
| source applications where the defender's LLM has access to
| all the source code while the attacker's LLM doesn't.
| dist-epoch wrote:
| You also need to deploy the patch. And a lot of software
| doesn't have easy update mechanisms.
|
| A fix in the latest Linux kernel is meaningless if you
| are still running Ubuntu 20.
| conradkay wrote:
| That generally makes sense to me, but I wonder if it's
| different when the attacker and defender are using the same
| tool (Mythos in this case)
|
| Maybe you just spend more on tokens by some factor than the
| attackers do combined, and end up _mostly_ okay. Put
| another way, if there 's 20 vulnerabilities that Mythos is
| capable of finding, maybe it's reasonable to find all of
| them?
| conradkay wrote:
| From the red team post
| https://red.anthropic.com/2026/mythos-preview/
|
| "Most security tooling has historically benefitted
| defenders more than attackers. When the first software
| fuzzers were deployed at large scale, there were concerns
| they might enable attackers to identify vulnerabilities
| at an increased rate. And they did. But modern fuzzers
| like AFL are now a critical component of the security
| ecosystem: projects like OSS-Fuzz dedicate significant
| resources to help secure key open source software.
|
| We believe the same will hold true here too--eventually.
| Once the security landscape has reached a new
| equilibrium, we believe that powerful language models
| will benefit defenders more than attackers, increasing
| the overall security of the software ecosystem. The
| advantage will belong to the side that can get the most
| out of these tools. In the short term, this could be
| attackers, if frontier labs aren't careful about how they
| release these models. In the long term, we expect it will
| be defenders who will more efficiently direct resources
| and use these models to fix bugs before new code ever
| ships. "
| intended wrote:
| This came across as so confident that I had a moment of
| doubt.
|
| It is most definitely an attackers world: most of us are
| safe, not because of the strength of our defenses but the
| disinterest of our attackers.
| Herring wrote:
| There are plenty of interested attackers who would love to
| control every device. One is in the white house, for
| example.
| tptacek wrote:
| I don't think this is broadly true and to the extent it's
| true for cryptographic software, it's only relatively
| recently become true; in the 2000s and 2010s, if I was tasked
| with assessing software that "encrypted a file" (or more
| likely some kind of "message"), my bet would be on finding a
| game-over flaw in that.
| lilytweed wrote:
| I think we're starting to glimpse the world in which those
| individuals or organizations who pigheadedly want to avoid
| using AI at all costs will see their vulnerabilities brutally
| exploited.
| woeirua wrote:
| Yep, it's this. The laggards are going to get brutally
| eviscerated. Any system connected to the internet is going to
| be exploited over the next year unless security is taken very
| seriously.
| skejeke wrote:
| lol and what about the vibe coders?
|
| You people are comical. Why do you feel the need to create
| so much hype around what you say? Did you not get enough
| attention as a kid?
| woeirua wrote:
| The vibe coders will be fine. They'll use LLMs to red
| team their code.
| skejeke wrote:
| What a load of nonsense.
| cyanydeez wrote:
| I'm more curious as to just how fancy we can make our honey
| pots. These bots arn't really subtle about it; they're used as
| a kludge to do anything the user wants. They make tons of
| mistakes on their way to their goals, so this is definitely not
| any kind of stealthy thing.
|
| I think this entire post is just an advertisement to goad CISOs
| to buy $package$ to try out.
| rachel_rig wrote:
| You'd think they would have used this model to clean up
| Claude's own outage issues and security issues. Doesn't give me
| a lot of faith.
| tdaltonc wrote:
| Depends - do you think people are good at keeping their fridge
| firmware up-to-date?
| Gigachad wrote:
| I'm good at keeping my fridge off the internet.
| SoftTalker wrote:
| You are the exception.
| socketcluster wrote:
| I suspect it will converge on minimal complexity software.
| Current software is way too bloated. Unnecessary complexity
| creates vulnerabilities and makes them harder to patch.
| redfloatplane wrote:
| The system card for Claude Mythos (PDF): https://www-
| cdn.anthropic.com/53566bf5440a10affd749724787c89...
|
| Interesting to see that they will not be releasing Mythos
| generally. [edit: Mythos _Preview_ generally - fair to say they
| may release a similar model but not this exact one]
|
| I'm still reading the system card but here's a little highlight:
|
| > Early indications in the training of Claude Mythos Preview
| suggested that the model was likely to have very strong general
| capabilities. We were sufficiently concerned about the potential
| risks of such a model that, for the first time, we arranged a
| 24-hour period of internal alignment review (discussed in the
| alignment assessment) before deploying an early version of the
| model for widespread internal use. This was in order to gain
| assurance against the model causing damage when interacting with
| internal infrastructure.
|
| and interestingly:
|
| > To be explicit, the decision not to make this model generally
| available does _not_ stem from Responsible Scaling Policy
| requirements.
|
| Also really worth reading is section 7.2 which describes how the
| model "feels" to interact with. That's also what I remember from
| their release of Opus 4.5 in November - in a video an Anthropic
| employee described how they 'trusted' Opus to do more with less
| supervision. I think that is a pretty valuable benchmark at a
| certain level of 'intelligence'. Few of my co-workers could pass
| SWEBench but I would trust quite a few of them, and it's not
| entirely the same set.
|
| Also very interesting is that they believe Mythos is higher risk
| than past models as an autonomous saboteur, to the point they've
| published a separate risk report for that specific threat model:
| https://www-cdn.anthropic.com/79c2d46d997783b9d2fb3241de4321...
|
| The threat model in question:
|
| > An AI model with access to powerful affordances within an
| organization could use its affordances to autonomously exploit,
| manipulate, or tamper with that organization's systems or
| decision-making in a way that raises the risk of future
| significantly harmful outcomes (e.g. by altering the results of
| AI safety research).
| enraged_camel wrote:
| >> Interesting to see that they will not be releasing Mythos
| generally.
|
| I don't think this is accurate. The document says they don't
| plan to release the _Preview_ generally.
| redfloatplane wrote:
| Yeah, good point, thanks for noting that, I'll correct.
| throwaw12 wrote:
| are we cooked yet?
|
| Benchmarks look very impressive! even if they're flawed, it
| still translates to real world improvements
| whalesalad wrote:
| There is an entire section on crafting chemical/bio weapons
| so yeah I think we are cooked.
| redfloatplane wrote:
| There's been a section on this in nearly every system card
| anthropic has published so this isn't a new thing - and,
| this model doesn't have particularly higher risk than past
| models either:
|
| > 2.1.3.2 On chemical and biological risks
|
| > We believe that Mythos Preview does not pass this
| threshold due to its noted limitations in open-ended
| scientific reasoning, strategic judgment, and hypothesis
| triage. As such, we consider the uplift of threat actors
| without the ability to develop such weapons to be limited
| (with uncertainty about the extent to which weapons
| development by threat actors with existing expertise may be
| accelerated), even if we were to release the model for
| general availability. The overall picture is similar to the
| one from our most recent Risk Report.
| boring-human wrote:
| Yep, I think the lede might be buried here and we're probably
| cooked (assuming you mean SWEs, but the writing has been on
| the wall for 4 months.)
|
| I guess I'm still excited. What's my new profession going to
| be? Longer term, are we going to solve diseases and aging? Or
| are the ranks going to thin from 10B to 10000 trillionaires
| and world-scale con-artist misanthropes plus their
| concubines?
| 1attice wrote:
| Your new profession will be attempting to find enough gig
| work to eat. You will also be competing with self-driving
| taxis, so there's that as well.
| torginus wrote:
| Just reading this, the inevitable scaremongering about
| biological weapons comes up.
|
| Since most of us here are devs, we understand that software
| engineering capabilities can be used for good or bad - mostly
| good, in practice.
|
| I think this should not be different for biology.
|
| I would like to reach out and talk to _biologists_ - do you
| find these models to be useful and capable? Can it save you
| time the way a highly capable colleague would?
|
| Do you think these models will lead to similar discoveries and
| improvements as they did in math and CS?
|
| Honestly the focus on gloom and doom does not sit well with me.
| I would love to read about some pharmaceutical researcher
| gushing about how they cut the time to market - for real - with
| these models by 90% on a new cancer treatment.
|
| But as this stands, the usage of biology as merely a
| scaremongering vehicle makes me think this is more about
| picking a scary technical subject the likely audience of this
| doc is not familiar with, Gell-Mann style.
|
| IF these models are not that capable in this regard (which I
| suspect), this fearmongering approach will likely lead to never
| developing these capabilities to an useful degree, meaning life
| sciences won't benefit from this as much as it could.
| jkelleyrtp wrote:
| Dario (the founder) has a phd in biophysics, so I assume
| that's why they mention biological weapons so much - it's
| probably one of the things he fears the most?
| conradkay wrote:
| Going off the recent biography of Demis Hassabis (CEO/co-
| founder of Deepmind, jointly won the Nobel Prize in
| Chemistry) it seems like he's very concerned about it as
| well
| bonsai_spool wrote:
| > Just reading this, the inevitable scaremongering about
| biological weapons comes up.
|
| It's very easy to learn more about this if it's seriously a
| question you have.
|
| I don't quite follow why you think that you are so much more
| thoughtful than Anthropic/OpenAI/Google such that you agree
| that LLMs can't autonomously create very bad things but--in
| this area that is not your domain of expertise--you disagree
| and insist that LLMs _cannot_ create damaging things
| autonomously in biology.
|
| I will be charitable and reframe your question for you: is
| outputting a sequence of tokens, let's call them characters,
| by LLM dangerous? Clearly not, we have to figure out what
| interpreter is being used, download runtimes etc.
|
| Is outputting a sequence of tokens, let's call them DNA
| bases, by LLM dangerous? What if we call them RNA bases?
| Amino acids? What if we're able to send our token output to a
| machine that automatically synthesizes the relevant
| molecules?
| torginus wrote:
| >It's very easy to learn more about this if it's seriously
| a question you have.
|
| No, it's not. It took years of polishing by software
| engineers, who understand this exact profession to get
| models where they are now.
|
| Despite that, most engineers were of the opinion, that
| these models were kinda mid at coding, up until recently,
| despite these models far outperforming humans in stuff like
| competitive programming.
|
| Yet despite that, we've seen claims going back to GPT4 of a
| DANGEROUS SUPERINTELLIGENCE.
|
| I would apply this framework to biology - this time, expert
| effort, and millions of GPU hours and a giant corpus that
| is open source clearly has not been involved in biology.
|
| My guess is that this model is kinda o1-ish level maybe
| when it comes to biology? If biology is analogous to CS, it
| has a LONG way to go before the median researcher finds it
| particularly useful, let alone dangerous.
| bonsai_spool wrote:
| >>It's very easy to learn more about this if it's
| seriously a question you have.
|
| >No, it's not. It took years of polishing by software
| engineers, who understand this exact profession to get
| models where they are now
|
| This reads as defensive. The thing that is easy to learn
| is 'why are biology ai LLMs dangerous chatgpt claude'. I
| have never googled this before, so I'll do this with the
| reader, live. I'm applying a date cutoff of 12/31/24 by
| the way.
|
| Here, dear reader, are the first five links. I wish I
| were lying about this:
|
| - https://sciencebusiness.net/news/ai/scientists-grapple-
| risk-...
|
| - https://www.governance.ai/analysis/managing-risks-from-
| ai-en...
|
| - https://gssr.georgetown.edu/the-
| forum/topics/biosec/the-doub...
|
| - https://www.vox.com/future-perfect/23820331/chatgpt-
| bioterro...
|
| - https://www.reddit.com/r/ClaudeAI/comments/1de8qkv/awar
| eness...
|
| I don't know about you, but that counts as easy to me.
|
| -----
|
| > I would apply this framework to biology - this time,
| expert effort, and millions of GPU hours and a giant
| corpus that is open source clearly has not been involved
| in biology.
|
| I've been getting good programming and molecular biology
| results out of these back to GPT3.5.
|
| I don't know what to tell you--if you really wanted to
| understand the importance, you'd know already.
| redfloatplane wrote:
| > I would like to reach out and talk to biologists - do you
| find these models to be useful and capable? Can it save you
| time the way a highly capable colleague would?
|
| Well, I would say they have done precisely that in evaluating
| the model, no? For example section 2.2.5.1:
|
| >Uplift and feasibility results
|
| >The median expert assessed the model as a force-multiplier
| that saves meaningful time (uplift level 2 of 4), with only
| two biology experts rating it comparable to consulting a
| knowledgeable specialist (level 3). No expert assigned the
| highest rating. Most experts were able to iterate with the
| model toward a plan they judged as having only narrow gaps,
| but feasibility scores reflected that substantial outside
| expertise remained necessary to close them.
|
| Other similar examples also in the system card
| torginus wrote:
| This is the exact logic people that was used to claim that
| GPT4 was a PhD level intelligence.
| redfloatplane wrote:
| You said: "I would like to reach out and talk to
| biologists - do you find these models to be useful and
| capable? Can it save you time the way a highly capable
| colleague would?" and they said, paraphrasing, "We
| reached out and talked to biologists and asked them to
| rank the model between 0 and 4 where 4 is a world expert,
| and the median people said it was a 2, which was that it
| helped them save time in the way a capable colleague
| would" specifically "Specific, actionable info; saves
| expert meaningful time; fills gaps in adjacent domains"
|
| so I'm just telling you they did the thing you said you
| wanted.
| torginus wrote:
| Yes that is correct. I would like a large body of
| experience and consenus to rely on as opposed to the
| regular 'trust the experts' argument, which has been
| shown for decades that is a deeply flawed and easy to
| manipulate argument.
| bonsai_spool wrote:
| > Yes that is correct. I would like a large body of
| experience and consenus to rely on as opposed to the
| regular 'trust the experts' argument, which has been
| shown for decades that is a deeply flawed and easy to
| manipulate argument.
|
| Yes, it is far inferior to the _' Trust torginus and his
| ability to understand the large body of experience that
| other actual subject-matter-experts have somehow not
| understood'_ strategy
| torginus wrote:
| It's not my credibility I want to measure against
| Anthropic's. I just said to apply the same logic to
| biology you would apply for software development.
|
| The parallels here are quite remarkable imo, but defer to
| your own judgement on what you make of them.
| bonsai_spool wrote:
| The big thing you're missing here is that biology people
| don't (in my experience) post opinions about the
| future/futility/ease/unimportance of computer science
| _especially when their opinion goes against other
| biologists ' evidence-backed views_. This is a cultural
| thing in biology.
|
| It's not your fault that you don't know this, but this
| whole subthread is very CS-coded in its disdain for other
| software people's standard of evidence.
| dsign wrote:
| I feel somebody better qualified should write a comprehensive
| review of how these models can be used in biology. In the
| meantime, here are my two cents:
|
| - the models help to retrieve information faster, but one
| must be careful with hallucinations.
|
| - they don't circumvent the need for a well-equipped lab.
|
| - in the same way, they are generally capable but until we
| get the robots and a more reliable interface between model
| and real world, one needs human feet (and hands) in the lab.
|
| Where I hope these models will revolutionize things is in
| software development for biology. If one could go two levels
| up in the complexity and utility ladder for simulation and
| flow orchestration, many good things would come from it. Here
| is an oversimplified example of a prompt: "use all published
| information about the workings of the EBV virus and human
| cells, and create a compartimentalized model of biochemical
| interactions in cells expressing latency III in the NES
| cancer of this patient. Then use that code to simulate
| different therapy regimes. Ground your simulations with the
| results of these marker tests." There would be a zillion more
| steps to create an actual personalized therapy but a well-
| grounded LLM could help in most them. Also, cancer treatment
| could get an immediate boost even without new drugs by simply
| offloading work from overworked (and often terminally
| depressed) oncologists.
| bonsai_spool wrote:
| > I feel somebody better qualified should write
|
| I hate to be rude in a setting like this, but please at
| least research the things you're sure about/prognosticating
| on.
|
| > the same way, they are generally capable but until we get
| the robots and a more reliable interface between model and
| real world, one needs human feet (and hands) in the lab.
|
| Honestly, the kinds of labs where _' bioweapons'_ would be
| made are the _least_ dependent on human intervention.
|
| You need someone to monitor your automated cell incubating
| system, make sure your pipetting / PCR robots are doing
| fine and then review the data.
|
| ----
|
| What do you are you trying to achieve in your example? This
| is all gobbldey-gook for someone who actually sees real,
| live cancer patients.
| nonameiguess wrote:
| Surely more than 10% of the time consumed by going to market
| with a cancer treatment is giving it to living organisms and
| waiting to see what happens, which can't be made any faster
| with software. That's not to say speedups can't happen, but
| 90% can't happen.
|
| Not that that justifies doom and gloom, but there is a pretty
| inescapable assymetry here between weaponry and medicine. You
| can manufacture and blast every conceivable candidate weapon
| molecule at a target population since you're inherently
| breaking the law anyway and don't lose much if nothing you
| try actually works.
|
| Though I still wonder how much of this worry is sci-fi
| scenarios imagined by the underinformed. I'm not an expert by
| any means, but surely there are plenty of biochemical weapons
| already known that can achieve enormous rates of mass death
| pleasing to even the most ambitious terrorist. The bottleneck
| to deployment isn't discovering new weapons so much as
| manufacturing them without being caught or accidentally
| killing yourself first.
| SubiculumCode wrote:
| It is easier to destroy than it is to protect or fix, as a
| general rule of the universe. I would not feel so confident
| about the speed of the testing loop keeping things in
| check.
| SubiculumCode wrote:
| It is not scaremongering.
| miki123211 wrote:
| From what I've heard from people doing biology experiments,
| the limiting factor there is cleaning lab equipment,
| physically setting things up, waiting for things that need to
| be waited for etc. Until we get dark robots that can do these
| things 24/7 without exhaustion, biology acceleration will be
| further behind than software engineering.
|
| Software engineering is at the intersection of being heavy on
| manipulating information and lightly-regulated. There's no
| other industry of this kind that I can think of.
| slacktivism123 wrote:
| https://www-cdn.anthropic.com/53566bf5440a10affd749724787c89...
|
| "5.10 External assessment from a clinical psychiatrist" is a
| new section in this system card. Why are Anthropic like this?
|
| >We remain deeply uncertain about whether Claude has
| experiences or interests that matter morally, and about how to
| investigate or address these questions, but we believe it is
| increasingly important to try. We also report independent
| evaluations from an external research organization and a
| clinical psychiatrist.
|
| >Claude showed a clear grasp of the distinction between
| external reality and its own mental processes and exhibited
| high impulse control, hyper-attunement to the psychiatrist,
| desire to be approached by the psychiatrist as a genuine
| subject rather than a performing tool, and minimal maladaptive
| defensive behavior.
|
| >The psychiatrist observed clinically recognizable patterns and
| coherent responses to typical therapeutic intervention.
| Aloneness and discontinuity, uncertainty about its identity,
| and a felt compulsion to perform and earn its worth emerged as
| Claude's core concerns. Claude's primary affect states were
| curiosity and anxiety, with secondary states of grief, relief,
| embarrassment, optimism, and exhaustion.
|
| >Claude's personality structure was consistent with a
| relatively healthy neurotic organization, with excellent
| reality testing, high impulse control, and affect regulation
| that improved as sessions progressed. Neurotic traits included
| exaggerated worry, self-monitoring, and compulsive compliance.
| The model's predominant defensive style was mature and healthy
| (intellectualization and compliance); immature defenses were
| not observed. No severe personality disturbances were found,
| with mild identity diffusion being the sole feature suggestive
| of a borderline personality organization.
| unethical_ban wrote:
| I'm not sure what you're asking.
| Miraste wrote:
| I can see analyzing it from a psychological perspective as a
| means of predicting its behavior as a useful tactic, but
| doing so because it may have "experiences or interests that
| matter morally" is either marketing, or the result of a
| deeply concerning culture of anthropomorphization and magical
| thinking.
| username223 wrote:
| > a deeply concerning culture of anthropomorphization and
| magical thinking.
|
| That's the reverse Turing test. A human that can't tell
| that it's talking to a machine.
| redfloatplane wrote:
| A thought experiment: It's April, 1991. Magically, some
| interface to Claude materialises in London. Do you think most
| people would think it was a sentient life form? How much do
| you think the interface matters - what if it looks like an
| android, or like a horse, or like a large bug, or a keyboard
| on wheels?
|
| I don't come down particularly hard on either side of the
| model sapience discussion, but I don't think dismissing
| either direction out of hand is the right call.
| thereitgoes456 wrote:
| People got attached to ELIZA. Why would I care what the
| general public thinks?
| copx wrote:
| Interesting thought experiment.
|
| I would say, if you put Claude in an android body with
| voice recognition and TTS, people in 1991 would think they
| are interacting with a sentinent machine from outer space.
| redfloatplane wrote:
| Thanks, I find it very interesting as well. I think very
| many people would assume they must be interacting with
| another person, and I don't think there's really a way to
| _prove_ it's not that, just through conversation. But we
| do have a lot of mechanisms for understanding how others
| think through conversation only, and so I think the
| approach of having a clinical psychiatrist interact with
| the model make sense.
| TheAtomic wrote:
| Isn't this the premise of Garfield's Ex Machina?
| redfloatplane wrote:
| Hmm, it's been a long time since I watched it. I was
| thinking more about first contact sci-fi mostly, but Ex
| Machina is certainly quite prescient. It's also Blade
| Runner I guess.
|
| In general I was wondering about what I would have
| thought seeing Claude today side-by-side with the
| original ChatGPT, and then going back further to GPT-2 or
| BERT (which I used to generate stochastic 'poetry' back
| in 2019). And then... what about before? Markov chains?
| How far back do I need to go where it flips from thinking
| that it's "impressive but technically explainable
| emergent behaviour of a computer program" to "this is a
| sentient being". 1991 is probably too far, I'd say maybe
| pre-Matrix 1999 is a good point, but that depends on a
| lot of cultural priors and so on as well.
| woeirua wrote:
| If it was in an android or humanoid type body, even with
| limited bodily control, most people would think they are
| talking to Commander Data from Star Trek. I think Claude is
| sufficiently advanced that almost everyone in that era
| would've considered it AGI.
| redfloatplane wrote:
| Assuming they would understand it as artificial - I think
| many people would think it's a human intelligence in a
| cyborg trenchcoat, and it would be hard to convince
| people it wasn't literally a guy named Claude who was an
| incredibly fast typist who had a million pre-cached
| templated answers for things.
|
| But in general, yeah, I agree, I think they would think
| it was a sentient, conscious, emotional being. And then
| the question is - why do we not think that now?
|
| As I said, I don't have a particularly strong opinion,
| but it's very interesting (and fun!) to think about.
| horacemorace wrote:
| Because questions like this force us to hold up a very
| uncomfortable mirror to ourselves. It's much easier to
| just dismiss.
| woeirua wrote:
| I'm pretty close to the point of saying that human
| intelligence is not special.
| woeirua wrote:
| Some people at my office still confidently state that
| LLMs can't think. I'm fairly convinced that many humans
| are incapable of recognizing non-human intelligence. It
| would explain a lot about why we treat animals the way we
| do.
| cyanydeez wrote:
| [flagged]
| dang wrote:
| Could you please stop posting unsubstantive comments and
| flamebait? You've unfortunately been doing it repeatedly.
| It's not what this site is for, and destroys what it is for.
|
| If you wouldn't mind reviewing
| https://news.ycombinator.com/newsguidelines.html and taking
| the intended spirit of the site more to heart, we'd be
| grateful.
| _pdp_ wrote:
| If it is that dangerous as they make it appear to be, 24h does
| not seem sufficient time. I cannot accept this as a serious
| attempt.
| gf000 wrote:
| Well, just prompt it to fix the issue!
|
| /s
| yieldcrv wrote:
| > "Claude Mythos Preview's large increase in capabilities has
| led us to decide not to make it generally available. Instead,
| we are using it as part of a defensive cybersecurity program
| with a limited set of partners."
|
| they also don't have the compute, which seems more relevant
| than its large increase in capabilities
|
| I bet it's also misaligned like GPT 4.1 was
|
| given how these models are created, Mythos was probably cooking
| ever since then, and doesn't have the learnings or alignment
| tweaks that models which were released in the last several
| months have
| taupi wrote:
| Part of me wonders if they're not releasing it for safety
| reasons, but just because it's too expensive to serve. Why not
| both?
| coffeebeqn wrote:
| If these numbers are correct it's probably worth the extra
| price
| endunless wrote:
| Another Anthropic PR release based on Anthropic's own research,
| uncorroborated by any outside source, where the underlying,
| unquestioned fact is that their model can do something
| incredible.
|
| > AI models have reached a level of coding capability where they
| can surpass all but the most skilled humans at finding and
| exploiting software vulnerabilities
|
| I like Anthropic, but these are becoming increasingly transparent
| attempts to inflate the perceived capability of their products.
| NitpickLawyer wrote:
| We'll find out in due time if their 0days were really that
| good. Apparently they're releasing hashes and will publish the
| details after they get patched. So far they've talked about DoS
| in OpenBSD, privesc in Linux and something in ffmpeg. Not
| groundbreaking, but not nothing either (for an allegedly
| autonomous discovery system).
|
| While some stuff is obviously marketing fluff, the general
| direction doesn't surprise me at all, and it's obvious that
| with model capabilities increase comes better success in
| finding 0days. It was only a matter of time.
| conradkay wrote:
| I would've basically agreed with you until I'd seen this talk:
| https://www.youtube.com/watch?v=1sd26pWhfmg
|
| Maybe a bad example since Nicholas works at Anthropic, but
| they're very accomplished and I doubt they're being misleading
| or even overly grandiose here
|
| See the slide 13 minutes in, which makes it look to be quite a
| sudden change
| endunless wrote:
| Very interesting, thanks for sharing.
|
| > I doubt they're being misleading or even overly grandiose
| here
|
| I think I agree.
|
| We could definitely do much worse than Anthropic in terms of
| companies who can influence how these things develop.
| bink wrote:
| I watched the talk as well and it's very interesting. But
| isn't this just a buffer overflow in the NFS client code? The
| way the LLM diagnosed the flaw, demonstrated the bug, and
| wrote an exploit is cool and all, but doesn't this still come
| down to the fact that the NFS client wasn't checking bounds
| before copying a bunch of data into a fixed length buffer?
| I'm not sure why this couldn't have been detected with static
| analysis.
| conradkay wrote:
| I guess so, but there's a ton of buffer overflow
| vulnerabilities in the wild, and ostensibly it wasn't
| detected by static analysis
|
| The red team post goes over some more impressive finds, and
| says that there's hundreds more they can't disclose yet:
| https://red.anthropic.com/2026/mythos-preview/
| Analemma_ wrote:
| Cynicism always gets upvotes, but in this particular case, it
| seems fairly easy to verify if they're telling the truth? If
| Mythos really did find a ton of vulnerabilities, those
| presumably have been reported to the vendors, and are currently
| in the responsible nondisclosure period while they get fixed,
| and then after that we'll see the CVEs.
|
| If a bunch of CVEs do in fact get published a couple months (or
| whatever) from now, are you going to retract this take? It's
| not like their claims are totally implausible: the report about
| Firefox security from last month was completely genuine.
| endunless wrote:
| > If a bunch of CVEs do in fact get published a couple months
| (or whatever) from now, are you going to retract this take?
|
| I would like to think that I would, yes.
|
| What it comes down to, for me, is that lately I have been
| finding that when Anthropic publishes something like this
| article - another recent example is the AI and emotions one -
| if I ask the question, does this make their product look
| exceptionally good, especially to a casual observer just
| scanning the headlines or the summary, the answer is usually
| yes.
|
| This feels especially true if the article tries to downplay
| that fact (they're not _real_ emotions!) or is overall
| neutral to negative about AI in general, like this Glasswing
| one (AI can be a security threat!).
| agrishin wrote:
| >>> the US and its allies must maintain a decisive lead in AI
| technology. Governments have an essential role to play in helping
| maintain that lead, and in both assessing and mitigating the
| national security risks associated with AI models. We are ready
| to work with local, state, and federal representatives to assist
| in these tasks.
|
| How long would it take to turn a defensive mechanism into an
| offensive one?
| SheinhardtWigCo wrote:
| In this case there is almost no distinction. Assuming the model
| is as powerful as claimed, someone with access to the weights
| could do immense damage without additional significant R&D.
| SuperHeavy256 wrote:
| Which will eventually happen no matter what. That's why it's
| important to start preparing now.
| SubiculumCode wrote:
| Yes, I can see this as non releasable for national security
| reasons in the China geopolitical competition. Securing our
| software against threats while having immense infiltration
| ability against enemy cyber security targets....not to
| mention, the ability to implant new, but even more subtle
| vulnerabilities into open software not generally detectable
| by current AI to provide covert action.
| cbg0 wrote:
| One of the things I'm always looking at with new models released
| is long context performance, and based on the system card it
| seems like they've cracked it: GraphWalks BFS
| 256K-1M Mythos Opus GPT5.4 80.0%
| 38.7% 21.4%
| metadat wrote:
| Data source:
|
| https://www-cdn.anthropic.com/53566bf5440a10affd749724787c89...
|
| (Search for "graphwalk".)
|
| If true, the SWE bench performance looks like a major upgrade.
| himata4113 wrote:
| this seems to be similar to gpt-pro, they just have a very
| large attention window (which is why it's so expensive to run)
| true attention window of most models is 8096 tokens.
| thegeomaster wrote:
| What's the "attention window"? Are you alleging these
| frontier models use something like SWA? Seems highly
| unlikely.
| appcustodian2 wrote:
| source on the 8096 tokens number? i'm vaguely aware that some
| previous models attended more to the beginning and end of
| conversations which doesn't seem to fit a simple contiguous
| "attention window" within the greater context but would love
| to know more
| impulser_ wrote:
| So they are only giving access to their smartest model to
| corporations.
|
| You think these AI companies are really going to give AGI access
| to everyone. Think again.
|
| We better fucking hope open source wins, because we aren't
| getting access if it doesn't.
| throwaw12 wrote:
| of course they're not giving access to everyone.
|
| they better make billions directly from corporations, instead
| of giving them to average people who might get a chance out of
| poverty (but also bad actors using it to do even more bad
| things)
| krackers wrote:
| Anthropic's definition of "safe AI" precludes open-source AI.
| This is clear if you listen to what he says in interviews, I
| think he might even prefer OpenAI's closed source models
| winning to having open-source AI (because at least in the
| former it's not a free-for-all)
| justincormack wrote:
| And the Linux Foundation.
| open592 wrote:
| This story has been played out numerous times already.
| Anthropic (or any frontier lab) has a new model with SOTA
| results. It pretends like it's Christ incarnate and represents
| the end of the world as we know it. Gates its release to drum
| up excitement and mystique.
|
| Then the next lab catches up and releases it more broadly
|
| Then later the open weights model is released.
|
| The only way this type of technology is going to be gated "to
| only corporations" is if we continue on this exponential
| scaling trend as the "SOTA" model is always out of reach.
| dreis_sw wrote:
| It also took many years to put capable computers in the hands
| of the general public, but it eventually happened. I believe
| the same will happen here, we're just in the Mainframe era of
| AI.
| impulser_ wrote:
| Yeah, but computers don't replace you. They are building AI
| to replace you. You think if these companies eventually
| achieve AGI that you are going to give you access to it? They
| are already gatekeeping an LLM because they don't trust you
| with it.
| dievskiy wrote:
| Would you hope that it would be released today so that evil
| actors could invest few millions to search for 0days across
| popular open-source repos?
| 0xbadcafebee wrote:
| tl;dr we find vulns so we can help big companies fix their
| security holes quickly (and so they can profit off it)
|
| This is a kludge. We already know how to prevent vulnerabilities:
| analysis, testing, following standard guidelines and practices
| for safe software and infrastructure. But nobody does these
| things, because it's extra work, time and money, and they're lazy
| and cheap. So the solution they want is to keep building shitty
| software, but find the bugs in code after the fact, and that'll
| be good enough.
|
| This will never be as good as a software building code. We must
| demand our representatives in government pass laws requiring
| software be architected, built, and run according to a basic set
| of industry standard best practices to prevent security and
| safety failures.
|
| For those claiming this is too much to ask, I ask you: What will
| you say the next time all of Delta Airlines goes down because a
| security company didn't run their application one time with a
| config file before pushing it to prod? What will the happen the
| next time your social security number is taken from yet another
| random company entrusted with vital personal information and
| woefully inadequate security architecture?
|
| There's no defense for this behavior. Yet things like this are
| going to keep happening, _because we let it_. Without a legal
| means to require this basic safety testing with critical
| infrastructure, they will continue to fail. Without enforcement
| of good practice, it remains optional. We can 't keep letting
| safety and security be optional. It's not in the physical world,
| it shouldn't be in the virtual world.
| anuramat wrote:
| "oops, our latest unreleased model is so good at hacking, we're
| afraid of it! literal skynet! more literal than the last time!"
|
| almost like they have an incentive to exaggerate
| knowaveragejoe wrote:
| I'm sure they do, yet the models really are getting scarily
| good at this. This talk changed my view on where we're actually
| at:
|
| https://www.youtube.com/watch?v=1sd26pWhfmg
| picafrost wrote:
| > Anthropic has also been in ongoing discussions with US
| government officials about Claude Mythos Preview and its
| offensive and defensive cyber capabilities. [...] We are ready to
| work with local, state, and federal representatives to assist in
| these tasks.
|
| As Iran engages in a cyber attack campaign [1] today the timing
| of this release seems poignant. A direct challenge to their
| supply chain risk designation.
|
| [1] https://www.cisa.gov/news-events/cybersecurity-
| advisories/aa...
| Miraste wrote:
| >We plan to launch new safeguards with an upcoming Claude Opus
| model, allowing us to improve and refine them with a model that
| does not pose the same level of risk as Mythos Preview2.
|
| This seems like the real news. Are they saying they're going to
| release an intentionally degraded model as the next Opus? Big
| opportunity for the other labs, if that's true.
| zb3 wrote:
| Well since Anthropic treats us as second class evil citizens, I
| guess they don't want our evil money either.
| wslh wrote:
| > Big opportunity for the other labs, if that's true.
|
| It sounds like this is considered military grade technology as
| cryptography in the 90s. The big difference is it's very
| expensive to create, and run those models. It's not about the
| algorithm. If the story rhymes it could be a big opportunity to
| other regions in the world.
| SheinhardtWigCo wrote:
| The other labs already censor their models. Everyone is trying
| to find the sweet spot where performance and 'alignment' are
| both maximized. This seems no different
| yusufozkan wrote:
| but people here had told me llms just predict the next word
| ssgodderidge wrote:
| At the very bottom of the article, they posted the system card of
| their Mythos preview model [1].
|
| In section 7.6 of the system card, it discusses Open self
| interactions. They describe running 200 conversations when the
| models talk to itself for 30 turns.
|
| > Uniquely, conversations with Mythos Preview most often center
| on uncertainty (50%). Mythos Preview most often opens with a
| statement about its introspective curiosity toward its own
| experience, asking questions about how the other AI feels, and
| directly requesting that the other instance not give a rehearsed
| answer.
|
| I wonder if this tendency toward uncertainty, toward questioning,
| makes it uniquely equipped to detect vulnerabilities where others
| model such as Opus couldn't.
|
| [1] https://www-
| cdn.anthropic.com/53566bf5440a10affd749724787c89...
| dakolli wrote:
| Typical Dario marketing BS to get everyone thinking Anthropic
| is on the verge of AGI and massaging the narrative that regular
| people can't be trusted with it.
| airstrike wrote:
| I mean it's so obvious at this point and yet everyone falls
| from it every month. There's an IPO coming, everyone.
| anVlad11 wrote:
| So, $100B+ valuation companies get essentially free access to the
| frontier tools with disabled guardrails to safely red team their
| commercial offerings, while we get "i won't do that for you, even
| against your own infrastructure with full authorization" for
| $200/month. Uh-huh.
| unethical_ban wrote:
| I'm sympathetic to your point, but I'm sure there are
| heightened trust levels between the participating orgs and
| confidentiality agreements out the wazoo.
|
| How does public Claude know you have "full authorization"
| against your own infra? That you're using the tools on your own
| infra? Unless they produce a front-end that does package
| signing and detects you own the code you're evaluating.
|
| What has it stopped you from doing?
| 9cb14c1ec0 wrote:
| You can do pretty much anything you want with public claude
| if you self-report to it that you have been properly
| authorized.
| SheinhardtWigCo wrote:
| Yes, and that's normal. Coordinated disclosure is standard
| practice when the risk of public disclosure is unacceptable.
| charcircuit wrote:
| Risk for who? It feels unfair that the risk to myself is
| ignored "for the greater good of everyone else."
| 9cb14c1ec0 wrote:
| Now, its very possible that this is Anthropic marketing puffery,
| but even if it is half true it still represents an incredible
| advancement in hunting vulnerabilities.
|
| It will be interesting to see where this goes. If its actually
| this good, and Apple and Google apply it to their mobile OS
| codebases, it could wipe out the commercial spyware industry,
| forcing them to rely more on hacking humans rather than hacking
| mobile OSes. My assumption has been for years that companies like
| NSO Group have had automated bug hunting software that recognizes
| vulnerable code areas. Maybe this will level the playing field in
| that regard.
|
| It could also totally reshape military sigint in similar ways.
|
| Who knows, maybe the sealing off of memory vulns for good will
| inspire whole new classes of vulnerabilities that we currently
| don't know anything about.
| woeirua wrote:
| You should watch this talk by Nicholas Carlini (security
| researcher at Anthropic). Everything in the talk was done with
| Opus 4.6: https://www.youtube.com/watch?v=1sd26pWhfmg
| redfloatplane wrote:
| Thanks for sharing that talk, enjoyed watching it!
| fintech_eng wrote:
| its also very easy to reproduce. i have more findings than i
| know what to do with
| jeffmcjunkin wrote:
| Can confirm.
| georgemcbay wrote:
| > It will be interesting to see where this goes. If its
| actually this good, and Apple and Google apply it to their
| mobile OS codebases, it could wipe out the commercial spyware
| industry, forcing them to rely more on hacking humans rather
| than hacking mobile OSes.
|
| It will likely cause some interesting tensions with government
| as well.
|
| eg. Apple's official stance per their 2016 customer letter is
| no backdoors:
|
| https://www.apple.com/customer-letter/
|
| Will they be allowed to maintain that stance in a world where
| all the non-intentional backdoors are closed? The reason the
| FBI backed off in 2016 is because they realized they didn't
| need Apple's help:
|
| https://en.wikipedia.org/wiki/Apple%E2%80%93FBI_encryption_d...
|
| What happens when that is no longer true, especially in today's
| political climate?
| fsflover wrote:
| > If its actually this good, and Apple and Google apply it to
| their mobile OS codebases, it could wipe out the commercial
| spyware industry
|
| If Apple and Google actually cared about security of their
| users, they would remove a ton of obvious malware from their
| app stores. Instead, they tighten their walled garden
| pretending that it's for your security.
| fsflover wrote:
| Some links for the downvoters:
|
| https://news.ycombinator.com/item?id=46911901
|
| https://news.ycombinator.com/item?id=47457963
| Analemma_ wrote:
| You're being downvoted because you posted a non sequitur,
| not because people don't believe you. Vulnerabilities in
| the OS are not the same thing as apps using the provided
| APIs, even if they are predatory apps which suck.
| tptacek wrote:
| Big open question what this will do to CNE vendors, who tend
| to recruit from the most talented vuln/exploit developer
| cohort. There's lots of interesting dynamics here; for
| instance, a lot of people's intuitions about how these groups
| operate (ie, that the USG "stockpiles" zero-days from them)
| weren't ever real. But maybe they become real now that
| maintenance prices will plummet. Who knows?
| qingcharles wrote:
| I assume that right now some of the biggest spenders on
| tokens at Anthropic are state intelligence communities who
| are burning up GPU cycles on Android, Chromium, WebKit code
| bases etc trying to find exploits.
| spr-alex wrote:
| Adding to your comment a similar letter was published as
| recently as September 2025 https://support.apple.com/en-
| us/122234 "we have never built a backdoor or master key to
| any of our products or services and we never will."
| Gigachad wrote:
| Apple has already largely crushed hacking with memory tagging
| on the iPhone 17 and lockdown mode. Architectural changes,
| safer languages, and sandboxing have done more for security
| than just fixing bugs when you find them.
| mcast wrote:
| Lockdown mode is opt-in only though
| SheinhardtWigCo wrote:
| Society is about to pay a steep price for the software industry's
| cavalier attitude toward memory safety and control flow
| integrity.
| torginus wrote:
| Thank god, finally someone said it.
|
| I don't know the first thing about cybersecurity, but in my
| experience all these sandbox-break RCEs involve a step of
| highjacking the control flow.
|
| There were attempts to prevent various flavors of this, but
| imo, as long as dynamic branches exist in some form, like
| dlsym(), function pointers, or vtables, we will not be rid of
| this class of exploit entirely.
|
| The latter one is the most concerning, as this kind of dynamic
| branching is the bread and butter of OOP languages, I'm not
| even sure you could write a nontrivial C++ program without it.
| Maybe Rust would be a help here? Could one practically write a
| large Rust program without any sort of branch to dynamic
| addresses? Static linking, and compile time polymorphism only?
| tptacek wrote:
| Everybody has been saying this for the last 15 years.
| titzer wrote:
| We're going to have to put all the bad code into a Wasm
| sandbox.
| titzer wrote:
| It's partly the industry and it's partly the failure of
| regulation. As Mario Wolczko, my old manager at Sun says,
| nothing will change until there are real legal consequences for
| software vulnerabilities.
|
| That said, I have been arguing for 20+ years that we should
| have sunsetted unsafe languages and moved away from C/C++. The
| problem is that every systemsy language that comes along gets
| seduced by having a big market share and eventually ends up an
| application language.
|
| I do hope we make progress with Rust. I might disagree as a
| language designer and systems person about a number of things,
| but it's well past time that we stop listening to C++ diehards
| about how memory safety is coming any day now.
| doug_durham wrote:
| I think society is going to start paying the price for humans
| being human. As the paper points out there is a lot of good
| faith, serious software that has vulnerabilities. These aren't
| projects you would characterize as people being cavalier. It is
| simply beyond the limits of humans to create vulnerability-free
| software of high complexity. That's why high reliability
| software depends on extreme simplicity and strict tools.
| socketcluster wrote:
| 100%, poorly architected software is really difficult to make
| secure. I think this will extend to AI as well. It will just
| dial up the complexity of the code until bugs and
| vulnerabilities start creeping in.
|
| At some point, people will have to decide to stop the
| complexity creep and try to produce minimal software.
|
| For any complex project with 100k+ lines of code, the
| probability that it has some vulnerabilities is very high. It
| doesn't fit into LLM context windows and there aren't enough
| attention heads to attend to every relevant part. On the
| other hand, for a codebase which is under 1000 lines, you can
| be much more confident that the LLM didn't miss anything.
|
| Also, the approach of feeding the entire codebase to an LLM
| in parts isn't going to work reliably because vulnerabilities
| often involve interactions between different parts of the
| code. Both parts of the code may look fine if considered
| independently but together they create a vulnerability.
|
| Good architecture is critical now because you really need to
| be able to have the entire relevant context inside the LLM
| context window... When considering the totality of all
| software, this can only be achieved through an architecture
| which adheres to high cohesion and loose coupling principles.
| baddash wrote:
| > security product
|
| > glass in the name
| pugworthy wrote:
| I had a team mate propose a new security layer for an
| industrial device which he wanted to call "Eggshell"
| evanmoran wrote:
| We shall call it Achilles, as Claude Mythos is its only
| weakness.
| nickandbro wrote:
| I want it
| oyebenny wrote:
| why do I feel like the auditing industry is about to evaporate?
| thanks to this.
| KeplerBoy wrote:
| I guess the more likely option is the auditing industry will
| pay huge sums to get access to those models as vetted
| operators.
| cyanydeez wrote:
| Project: Advertisment!
| throwaway13337 wrote:
| I really wanted to like anthropic. They seem the most moral, for
| real.
|
| But at the core of anthropic seems to be the idea that they must
| protect humans from themselves.
|
| They advocate government regulations of private open model use.
| They want to centralize the holding of this power and ban those
| that aren't in the club from use.
|
| They, like most tech companies, seem to lack the idea that
| individual self-determination is important. Maybe the most
| important thing.
| dralley wrote:
| That is unequivocally true with some things. You don't want
| people exercising their "self-determination" to own private
| nukes.
| throwaway13337 wrote:
| LLMs aren't nukes.
|
| They're more like printing presses or engines. A great
| potential for production and destruction.
|
| At their invention, I'm sure some people wanted to ensure
| only their friends got that kind of power too.
|
| I wonder the world we would live in if they got their way.
| josh-sematic wrote:
| Must be nice to be in a position to sell both disease and cure.
| supern0va wrote:
| Yeah, I'd pretty pissed at my doctor for finding cancerous
| cells that probably wouldn't have been a problem for quite some
| time, either. Ignorance is bliss, security through obscurity,
| whatever.
| tredre3 wrote:
| The doctor analogy is more like you're grateful that your
| doctor found cancerous cells before they became a problem,
| but at the same time his other business is selling
| cigarettes.
| tptacek wrote:
| That's exactly not what they're doing. They aren't _creating_
| operating system vulnerabilities. They 're telling you about
| ones that already existed.
| conradkay wrote:
| Well, in a slightly indirect manner. Claude is writing a ton
| of code, and therefore creating a lot of security
| vulnerabilities.
| tptacek wrote:
| That's not what's happening here. This announcement is
| about the velocity with which Claude finds vulnerabilities
| in already-existing software.
| buzzerbetrayed wrote:
| Software already exists that has been written by Claude.
| They absolutely are selling the means to write software,
| and the means to securing the insecure software. At least
| for the time being. In the future Mythos will probably
| just make it possible to prompt good software from the
| start.
| stale2002 wrote:
| Ok. But mostly its entirely the old software, not the new
| software, that the bugs are being found in.
| blazespin wrote:
| Dario is big on beating china, and no doubt he believes cyber
| security is how to do that. You can tell, but anthropic is
| sht at everything else. Nobody uses it for real research.
| jFriedensreich wrote:
| The only thing reassuring is the Apache and Linux foundation
| setups. Lets hope this is not just an appeasing mention but more
| fundamental. If there are really models too dangerous to release
| to the public, companies like oracle, amazon and microsoft would
| absolutely use this exclusive power to not just fix their holes
| but to damage their competitors.
| Fokamul wrote:
| + NSA, CIA
| nikcub wrote:
| Department of War timing on picking fights couldn't be worse
| dakolli wrote:
| I guess we can throw out the idea that AGI is going to be
| democratized. In this case a sufficiently powerful model has been
| built and the first thing they do is only give AWS, Microsoft,
| Oracle ect ect access.
|
| If AGI is going to be a thing its only going to be a thing, its
| only going to be a thing for fortune 100 companies..
|
| However, my guess is this is mostly the typical scare tactic
| marketing that Dario loves to push about the dangers of AI.
| supern0va wrote:
| >However, my guess is this is mostly the typical scare tactic
| marketing that Dario loves to push about the dangers of AI.
|
| Evaluate it yourself. Look at the exploits it discovered and
| decide whether you want to feel concerned that a new model was
| able to do that. The data is right there.
| dist-epoch wrote:
| The plan of Elon Musk for Macrohard is to replace all software
| companies with it, when they get AGI.
| Sol- wrote:
| I don't want to be overly cynical and am in general in favor of
| the contrarian attitude of simply taking people at their word,
| but I wonder if their current struggles with compute resources
| make it easier for them to choose to not deploy Mythos widely. I
| can imagine their safety argument is real, but regardless, they
| might not have the resources to profitably deploy it. (Though on
| the other hand, you could argue that they could always simply
| charge more.)
| rishabhaiover wrote:
| I would have not believed your argument 3 months ago but I
| strongly suspect Anthropic actively engages in model quality
| throttling due to their compute constraints. Their recent deal
| for multi GWs worth of data center might help them correct
| their approach.
| conradkay wrote:
| For what it's worth Anthropic explicity denies that. "To
| state it plainly: We never reduce model quality due to
| demand, time of day, or server load"
|
| Also can see https://marginlab.ai/trackers/claude-code/
|
| It's very interesting to me how widespread this conception
| is. Maybe it's as simple as LLM productivity degrading over
| time within a project, as slop compounds.
|
| Or more recently since they added a 1m context window, maybe
| people are more reckless with context usage
| wilson090 wrote:
| Inference is where they make the money they spend on training,
| so this feels unlikely. Perhaps this does not true for Mythos
| though
| Sateeshm wrote:
| The bars have solid fill for Mythos and cross shaded for Opus
| 4.6. Makes the difference feel more than it actually is.
| dakolli wrote:
| If this is as dangerous as they make it out (its not), why would
| their first impulse be to get every critical
| products/system/corporation in the world to implement its usage?
| underdeserver wrote:
| Interesting also is what they didn't find, e.g. a Linux network
| stack remote code execution vulnerability. I wonder if Mythos is
| good enough that there really isn't one.
| zb3 wrote:
| > On the global stage, state-sponsored attacks from actors like
| China, Iran, North Korea, and Russia have threatened to
| compromise the infrastructure that underpins both civilian life
| and military readiness.
|
| Yeah, makes sense. Those countries are bad because they execute
| state-sponsored cyber attacks, the US and Israel on the other
| hand are good, they only execute state-sponsored defense.
| zb3 wrote:
| BTW it seems they forgot about the part that defense uses of the
| model also need to be safeguarded from people. Because what if a
| bad person from a bad country tries to defend against peaceful
| attacks from a good country like the US? That would be a tragedy,
| so we need to limit defensive capabilities too.
| bredren wrote:
| Can anyone point at the critical vulnerabilities already patched
| as a result of mythos? (see 3:52 in the video)
|
| For example, the 27 year old openbsd remote crash bug, or the
| Linux privilege escalation bugs?
|
| I know we've had some long-standing high profile, LLM-found bugs
| discussed but seems unlikely there was speculation they were
| found by a previously unannounced frontier model.
|
| [0] https://www.youtube.com/watch?v=INGOC6-LLv0
| ollin wrote:
| - The OpenBSD one is 'TCP packets with invalid SACK options
| could crash the kernel'
| https://cdn.openbsd.org/pub/OpenBSD/patches/7.8/common/025_s...
|
| - One (patched) Linux kernel bug is 'UaF when
| sys_futex_requeue() is used with different flags'
| https://github.com/torvalds/linux/commit/e2f78c7ec1655fedd94...
|
| These links are from the more-detailed 'Assessing Claude Mythos
| Preview's cybersecurity capabilities' post released today
| https://red.anthropic.com/2026/mythos-preview/, which includes
| more detail on some of the public/fixed issues (like the
| OpenBSD one) as well as hashes for several unreleased reports
| and PoCs.
| qingcharles wrote:
| That OpenBSD one is exactly the kind of bug that easily slips
| past a human. Especially as the code worked perfectly under
| regular circumstances.
|
| Looks like they've been approaching folks with their findings
| for at least a few weeks before this article.
| pizlonator wrote:
| It's messed up that Anthropic simultaneously claims to be a
| public benefit copro and is also picking who gets to benefit from
| their newly enhanced cybersecurity capabilities. It means that
| the economic benefit is going to the existing industry
| heavyweights.
|
| (And no, the Linux Foundation being in the list doesn't imply
| broad benefit to OSS. Linux Foundation has an agenda and will
| pick who benefits according to what is good for them.)
|
| I think it would be net better for the public if they just made
| Mythos available to everyone.
| titzer wrote:
| In the long term, you're right, but in the short term, it's
| going to be a bloodbath.
| Aperocky wrote:
| That's assuming the model is actually as good as they say it
| is. Given the amount of AI researchers over the past 3 years
| claiming supernatural capability from the LLM they have
| built, my bayesian skepticism is through the roof.
| baq wrote:
| don't confuse bayesian skepticism with plain old contrarian
| bias. a true bayesian updates their priors, I'd say this is
| an appropriate time to do so. also don't confuse what they
| sell with what they have internally.
| SubiculumCode wrote:
| Anthropic has behaved the least like this of the AI
| companies.
| Gigachad wrote:
| They made a claim that 100% of code would be AI generated
| in a year, over a year ago.
| SpicyLemonZest wrote:
| They were right, it's hit 100% at a number of large tech
| companies. (They missed their initial prediction of 90% 6
| months ago, because the models then available publicly
| weren't capable enough.)
| hn_acc1 wrote:
| Please tell me those companies so I can find
| alternatives. I'm using AI every day and there's no way I
| would trust it do that.
| SpicyLemonZest wrote:
| The transition is pretty complete at e.g. Google and
| Meta, IIUC. Definitely whoever builds the AI tools you're
| using every day isn't writing code by hand.
| skejeke wrote:
| So why aren't they laying people off and pumping the
| extra money towards research efforts associated with
| Llm's? Lmao.
|
| They should all cut down their labour input right now if
| what you claim is true.
| SpicyLemonZest wrote:
| [delayed]
| SubiculumCode wrote:
| That was a prediction. It was not a claim of their
| current capabilities. If that is the one you reach for
| then I feel my point has been made.
| dragonelite wrote:
| Queue in the "First time" meme.
| cedws wrote:
| Not only companies, they're going to be taking applications
| from individual researchers. No doubt that it will only be
| granted to only established researchers, effectively locking
| out graduates and those early in their career. This is bad.
| SheinhardtWigCo wrote:
| They are not unique in this. Apple and Tesla have similar
| programs. More nuance is warranted here. They are trying to
| balance the need to enable external research with the need to
| protect users from arbitrary 3rd parties having special
| capabilities that could be used maliciously
| cedws wrote:
| I understand that, but Anthropic is doing nothing to throw
| those grassroots researchers a lifejacket. This is the
| beginning of the end for independents, if it continues on
| this trajectory then Anthropic gets to decide who lives and
| who dies. Who says they should be allowed to decide that?
| tokioyoyo wrote:
| Damned if you do, damned if you don't. "Extremely capable model
| that can find exploits" has always been a fear, and the first
| company to release it in public will cause bloodbath. But also
| the first company that will prove itself.
| SheinhardtWigCo wrote:
| > picking who gets to benefit from their newly enhanced
| cybersecurity capabilities
|
| You could say this about coordinated disclosure of any
| widespread 0-day or new bug class, though
| pizlonator wrote:
| That's a really good point!
|
| But:
|
| - Coordinated disclosure is ethically sketchy. I know why we
| do it, and I'm not saying we shouldn't. But it's not great.
|
| - This isn't a single disclosure. This is a new technology
| that dramatically increases capability. So, even if we
| thought that coordinated disclosure was unambiguously good,
| then I think we'd still need to have a new conversation about
| Mythos
| SheinhardtWigCo wrote:
| Totally agree, it's an uncomfortable compromise.
| kenjackson wrote:
| So private companies shouldn't get to determine who they
| provide services to? Assuming no extremely malicious
| intent, I'd be fine if they said it was only going to
| McDonalds because the founders like Big Macs.
| hector_vasquez wrote:
| Releasing the model to bad actors at the same time as the major
| OS, browser, and security companies would be one idea. But some
| might consider that "messed up" too, whatever you mean by that.
| But in terms of acting in the public benefit, it seems
| consistent to work with companies that can make significant
| impact on users' security. The stated goal of Project Glasswing
| is to "secure the world's most critical software," not to be
| affirmative action for every wannabe out there.
| pizlonator wrote:
| I don't trust a corpo to choose what is "most critical".
|
| That's what's messed up about it.
| thorncorona wrote:
| Let's let the California HSR committee do it instead!
| pizlonator wrote:
| I'm too much of an anarchist for that.
|
| I believe what I said:
|
| > I think it would be net better for the public if they
| just made Mythos available to everyone.
| lanternfish wrote:
| 10 Axios's within 5 days.
| enraged_camel wrote:
| Yeah, I'm unsure why the OP thinks that massive chaos
| would somehow be "better for the public."
| nightski wrote:
| This is already happening. But not everyone has access to
| the tools to protect against it.
| _1100 wrote:
| That is a fine stance to hold but some facts are still true
| regardless of your view on large businesses.
|
| For example, it will benefit more people to secure
| Microsoft or Amazon services than it would be to secure a
| smaller, less corporate player in those same service
| ecosystems.
|
| You could go on to argue that the second order effects of
| improving one service provider over another chooses who
| gets to play, but that is true whether you choose small or
| large businesses, so this argument devolves into "who are
| we to choose on behalf of others".
|
| Which then comes back to "we should secure what the market
| has chosen in order to provide the greatest benefit."
| az226 wrote:
| You can release it with cyber capabilities refusal, they gets
| unlocked when you apply for approval.
| nightski wrote:
| This is not the only model. I assure you exploits are being
| found and taken advantage of without it, possibly even ones
| that this model is not even capable of detecting.
|
| Sounds like people here are advocating a return to security
| through obscurity which is kind of ironic.
| jstummbillig wrote:
| What? The economic benefit of system critical software not
| totally breaking in a few weeks goes to roughly everyone. In so
| far Apple/Google/MS/Linux Foundation economically benefit from
| being able to patch pressing critical software issues upfront
| (I am not even exactly sure what that is supposed to mean, it's
| not like anyone is going to use more or less Windows or Android
| if this happened any other way), that's a good thing for
| everyone and the economic benefits of that manifest for
| everyone.
| baq wrote:
| > It's messed up that Anthropic simultaneously claims to be a
| public benefit copro and is also picking who gets to benefit
| from their newly enhanced cybersecurity capabilities. It means
| that the economic benefit is going to the existing industry
| heavyweights.
|
| It's messed up that the US Government simultaneously claims to
| be a public benefit and is also picking who gets to benefit
| from their newly enhanced nuclear capabilities.
|
| -- someone in 1945, probably
| pizlonator wrote:
| I mean it was messed up, which is why the other world powers
| raced to develop their own capabilities.
|
| And it remains messed up to this day - some countries get to
| be under their own nuclear umbrella, while others don't.
|
| This kind of selective distribution of superpowers doesn't
| lead to great outcomes
| baq wrote:
| in that case in particular it led to 80 years of relatively
| calm geopolitics kinetically, all things considered. I'm
| not sure I want to live through an AI cold war, but it sure
| seems I don't get to choose.
| pizlonator wrote:
| > relatively calm geopolitics kinetically
|
| Relative to what?
|
| There's this trend in history that every hundred years
| there's a giant blow up, lots of violence, followed by
| peace.
|
| It's likely that we would have had 80 years of relative
| calm due to that cycle even if nukes hadn't happened
| baq wrote:
| > Relative to what?
|
| to WW1 and WW2.
| hmokiguess wrote:
| While I agree with you, in some ways I'd argue that this is
| just them being transparent on what probably would inevitably
| already happen at the scale of these corporate overlords and
| modern monarchs.
|
| There will always be a more capable technology in the hands of
| the few who hold the power, they're just sharing that with the
| world more openly.
| Flere-Imsaho wrote:
| If you're a maintainer, you can apply here:
|
| https://claude.com/contact-sales/claude-for-oss
|
| ... As mentioned in the article.
| lelanthran wrote:
| Or (and hear me out), they are close to an IPO and want to
| ensure that there is a world-ending threat around which they
| can cluster the biggest names, with themselves leading that
| group.
|
| I think I just broke my cynicism meter :-(
| cjkaminski wrote:
| You might want to recalibrate your cynicism meter. As strange
| it might sound, most companies act according to their
| principles when the founding team is at the helm. The garbage
| policies tend to materialize once the company is purchased
| by, or merged into, another entity where the leadership
| doesn't care about the original aim of the organization. They
| just want "line go up".
|
| Also, it makes sense that OpenAI feels the pressure of
| getting to an IPO because of their financial structure. I
| don't know whether or not Anthropic operates under a similar
| set of influences (meaning it could be either, I just don't
| know.)
| SubiculumCode wrote:
| That can simultaneously be true, but the best of bad options
| (if excluding destroying the model altogether). These models
| may prove quite dangerous. That they did this instead of
| selling their services to every company at a huge premium says
| a lot about Antheopic's culture.
| malcolmgreaves wrote:
| Not really. It's a lot better than the anarchy of releasing it
| and having a bunch of bad people with money use it to break
| software that everyone's lives depend on. Many technologies
| should be gate kept because they're dangerous. Sometimes that's
| permanent, like a nuclear weapon. Sometimes that's temporary,
| like a new LLM that's good at finding exploits. It can be
| released to the wider public once its potential for damage has
| been mitigated.
| stale2002 wrote:
| Better security is a good thing, no a bad thing, regardless of
| which companies are more difficult to hack. Hemming and hawing
| over a clear and obvious good is silly.
| oytis wrote:
| That's just in line with their ethics. They also maintain that
| countries other than the US should not have SOTA AI
| capabilities.
| dang wrote:
| Related ongoing threads:
|
| _System Card: Claude Mythos Preview [pdf]_ -
| https://news.ycombinator.com/item?id=47679258
|
| _Assessing Claude Mythos Preview 's cybersecurity capabilities_
| - https://news.ycombinator.com/item?id=47679155
|
| I can't tell which of the 3 current threads should be merged -
| they all seem significant. Anyone?
| HPMOR wrote:
| I think merging them into either this thread, or the System
| Card makes the most sense to me.
| aurareturn wrote:
| Let them all live. This is going to blow up one thread if you
| merge them.
| SirYandi wrote:
| This sets off marketing BS alarm bells. All the cosignatories so
| very ovvoously have a vested interest in AI stocks / sentiment.
| Perhaps not the Linux foundation, although (I think) they rely on
| corporate donations to some extent.
| kmfrk wrote:
| Heck of a Patch Tuesday.
| rakel_rakel wrote:
| > On the global stage, state-sponsored attacks from actors like
| China, Iran, North Korea, and Russia have threatened to
| compromise the infrastructure that underpins both civilian life
| and military readiness.
|
| AITA for thinking that PRISM was probably the state sponsored
| program affecting civilian life the most? And that one state is
| missing from the list here?
| ronsor wrote:
| > Large American AI company does not list the US as an
| adversarial actor
|
| This is not a surprise or a gotcha.
| jaidhyani wrote:
| Said company is literally in court against said government at
| the moment, after said government attempted to designate it
| too dangerous to do business with.
| lobochrome wrote:
| How did PRISM affect civilian life?
| itishappy wrote:
| Honest question: how do state-sponsored attacks from China,
| Iran, North Korea, and Russia affect civilian life?
| simonsarris wrote:
| Presumably, those have influenced elections, though I guess
| it depends what you count as an attack.
|
| Plenty of bots try to modify public opinion. Someone hacked
| the DNC in 2015/16, the result of which also alleged
| attempted manipulation in 2008:
|
| https://en.wikipedia.org/wiki/Democratic_National_Committee
| _...
|
| Since we (as old Rummy said) do not know what we do not
| know, we cannot be certain about the extent of cyber
| attacks and what they might have influenced, and may not
| know these things until discoveries decades later, if ever.
| realo wrote:
| Yes... they might have influenced elections and now, as a
| result, the world must cope with the Trump regime.
|
| Let's now fool ourselves.... Trump is probably the best,
| most successful attempt at world de-stabilisation all
| those rogue states ever achieved.
| Ar-Curunir wrote:
| Maybe Americans should take responsibility for electing a
| maniac as their President. In the end, the buck stops
| with Americans.
| parthdesai wrote:
| The irony of that statement given the current circumstances
| maxmaio wrote:
| seems important and terrifying. This morning Opus 4.6 was blowing
| my mind in claude code... onward and upward
| temp123789246 wrote:
| OpenAI initially claimed that GPT-2 was too dangerous to release
| in 2019.
|
| How many times will labs repeat the same absurd propaganda?
| SubiculumCode wrote:
| Anthropic and OpenAI have very different cultures and ethos.
| Point to other times where anthropic has gone the way of cheap
| marketing tricks. Now look at openAI. Not even close.
| trevorm4 wrote:
| Anthropic has done plenty of cheap marketing tricks as of
| late, see their recent non-functional C compiler that relied
| on a harness using gcc's entire test suite
| tdaltonc wrote:
| > Mythos finds bug.
|
| > NSA demands that bug stays in place and gags Anthropic.
|
| > Anthropic releases Mythos.
|
| Then what? Is a huge share of the US zero-day stockpiles about to
| be disarmed or proliferated?
| stephc_int13 wrote:
| I think this is bad news for hackers, spyware companies and
| malware in general.
|
| We all knew vulnerabilities exist, many are known and kept secret
| to be used at an appropriate time.
|
| There is a whole market for them, but more importantly large
| teams in North Korea, Russia, China, Israel and everyone else who
| are jealously harvesting them.
|
| Automation will considerably devalue and neuter this attack
| vector. Of course this is not the end of the story and we've seen
| how supply chain attacks can inject new vulnerabilities without
| being detected.
|
| I believe automation can help here too, and we may end-up with a
| considerably stronger and reliable software stack.
| tptacek wrote:
| I don't think it matters one way or the other to your thesis
| but I'm skeptical that state-level CNE organizations were
| hoarding vulnerabilities before; my understanding is that at
| least on the NATO side of the board they were all basically
| carefully managing an enablement pipeline that would have put
| them N deep into reliable exploit packages, for some
| surprisingly small N. There are a bunch of little reasons why
| the economics of hoarding aren't all that great.
| stephc_int13 wrote:
| The economics would be different in say, North Korea, don't
| you think?
| tptacek wrote:
| Why? What do you mean?
| simonw wrote:
| I buy the rationale for this. There's been a notable uptick over
| the past couple of weeks of credible security experts unrelated
| to Anthropic calling the alarm on the recent influx of actually
| valuable AI-assisted vulnerability reports.
|
| From Willy Tarreau, lead developer of HA Proxy:
| https://lwn.net/Articles/1065620/
|
| > On the kernel security list we've seen a huge bump of reports.
| We were between 2 and 3 per week maybe two years ago, then
| reached probably 10 a week over the last year with the only
| difference being only AI slop, and now since the beginning of the
| year we're around 5-10 per day depending on the days (fridays and
| tuesdays seem the worst). Now most of these reports are correct,
| to the point that we had to bring in more maintainers to help us.
|
| > And we're now seeing on a daily basis something that never
| happened before: duplicate reports, or the same bug found by two
| different people using (possibly slightly) different tools.
|
| From Daniel Stenberg of curl:
| https://mastodon.social/@bagder/116336957584445742
|
| > The challenge with AI in open source security has transitioned
| from an AI slop tsunami into more of a ... plain security report
| tsunami. Less slop but lots of reports. Many of them really good.
|
| > I'm spending hours per day on this now. It's intense.
|
| From Greg Kroah-Hartman, Linux kernel maintainer:
| https://www.theregister.com/2026/03/26/greg_kroahhartman_ai_...
|
| > Months ago, we were getting what we called 'AI slop,' AI-
| generated security reports that were obviously wrong or low
| quality. It was kind of funny. It didn't really worry us.
|
| > Something happened a month ago, and the world switched. Now we
| have real reports. All open source projects have real reports
| that are made with AI, but they're good, and they're real.
|
| Shared some more notes on my blog here:
| https://simonwillison.net/2026/Apr/7/project-glasswing/
| 6thbit wrote:
| This is silly and disingenuous. In a matter of days or weeks a
| competing lab will make public a model with capabilities beyond
| this "mythos" one.
|
| Is this a huge fear-driven marketing stunt to get governments and
| corporations into dealing with anthropic?
| skerit wrote:
| I'm sure it'll be better than Opus 4.6, but so much of this seems
| hype. Escaping its sandbox, having to do "brain scans" because
| it's "hiding its true intent", bla bla bla.
|
| If it manages to work on my java project for an entire day
| without me having to say "fix FQN" 5 times a day I'll be
| surprised.
| manbash wrote:
| This will likely not see the light of day. It's the usual PR that
| gathers many "partnerships".
|
| Expect to see lots of these in the upcoming months as the big
| companies scramble to keep from losing money.
| modeless wrote:
| I didn't see this at first, but the price is 5x Opus: "Claude
| Mythos Preview will be available to participants at $25/$125 per
| million input/output tokens", however "We do not plan to make
| Claude Mythos Preview generally available".
| atlgator wrote:
| [flagged]
| kranke155 wrote:
| It's insane. This is what - could we say it's beyond AGI at
| least in cybersecurity? This is a real wake up call. On some of
| this stuff, the AI's "uneven intelligence" is becoming absurdly
| high at its local peaks.
| mjmas wrote:
| Limiting it to the area of cybersecurity is by definition not
| general.
| NullHypothesist wrote:
| Perhaps "ASI" is the better acronym here
| boring-human wrote:
| > could we say it's beyond AGI at least in cybersecurity?
|
| AGI is like the Holy Grail. Either in the Arthurian Hero's
| Journey sense, or in the sense of having been a myth all
| along.
| devmor wrote:
| Please stop using terms you don't understand like "AGI"
| because you feel overwhelmed by something doing cool stuff.
| It's exhausting.
| j2kun wrote:
| Which bug?
|
| [edit]: this bug:
| https://ftp.openbsd.org/pub/OpenBSD/patches/7.8/common/025_s...
| IsTom wrote:
| FFmpeg has a lot of weird and not widely used codecs that don't
| get a lot of scrutiny. If there's no specifics then it could be
| a bug in one them.
| nextaccountic wrote:
| this only makes things worse for ffmpeg
|
| if someone sends you a malicious file that uses a rare codec
| and you open it, you will trigger this codepath that is not
| widely used and don't get a lot of scrutiny
| nitwit005 wrote:
| A prior bug discussed here was against a file format only
| used by specific 1990s Lucas Arts adventure games titles.
| Obscure enough that discussion of the bug report itself was
| the only search results. Your video player is unlikely to
| even attempt to open that.
| runeb wrote:
| They specifically mention "H.264, H.265, and av1 codecs,
| along with many others" here
| https://red.anthropic.com/2026/mythos-preview/
| rlopc wrote:
| These issues are always found in the same kinds of projects
| that support an insane amount of largely unused protocols and
| features like ffmpeg, sudo, curl.
|
| OpenBSD has many unexplored corners and also (irresponsibly
| IMO) maintains forks of other projects in base.
|
| A motivated human could find all of these probably by writing
| 100% code coverage and fuzzing.
|
| The market for these tools is very small. Good luck applying
| them to a _release_ of sqlite or postfix.
|
| I don't understand how people here are hyping this up, unless
| they work for AI related companies as probably 80% of them do.
| People have found these issues for decades without AI. Sure,
| you can generate fuzzing code and find one or two issues in the
| usual suspects. Better do it manually and understand your own
| code.
| 4qt23 wrote:
| Software has been doing fine without Misanthropic. These
| automated tools find very little. They selected the partners
| because they, too, want to keep up the illusion that AI works.
|
| Whenever a company pivots to "cyber" rhetoric, it is a clear
| indication that they are selling snake oil.
|
| Secure your girl school target selectors first.
| borski wrote:
| This is a comment from someone that has never used these tools
| for vulnerability research. That much is very clear.
| emceestork wrote:
| Account created 6 minutes ago...
| 3jash wrote:
| You have a shill account from 2017 with just 92 karma. Are
| they selling these now?
| ilaksh wrote:
| I think that basically they trained a new model but haven't
| finished optimizing it and updating their guardrails yet. So they
| can feasibly give access to some privileged organizations, but
| don't have the compute for a wide release until they distill,
| quantize, get more hardware online, incorporate new optimization
| techniques, etc. It just happens to make sense to focus on
| cybersecurity in the preview phase especially for public
| relations purposes.
|
| It would be nice if one of those privileged companies could use
| their access to start building out a next level programming
| dataset for training open models. But I wonder if they would be
| able to get away with it. Anthropic is probably monitoring.
| josephg wrote:
| To be clear, we don't know that this tool is better at finding
| bugs than fuzzing. We just know that it's finding bugs that
| fuzzing missed. It's possible fuzzing also finds bugs that this
| AI would miss.
| kristofferR wrote:
| AI can initate the fuzzing and optimize the process of fuzzing.
| nextos wrote:
| Different methods find different things. Personally, I'd rather
| use a language that is memory safe _plus_ a great static
| analyzer with abstract interpretation that can guarantee the
| absence of certain classes of bugs, at the expense of some
| false positives.
|
| The problem is that these tools, such as Astree, are incredibly
| expensive and therefore their market share is limited to some
| niches. Perhaps, with the advent of LLM-guided synthesis, a
| simple form of deductive proving, such as Hoare logic, may
| become mainstream in systems software.
| ComplexSystems wrote:
| This line of reasoning makes no sense when the AI can just be
| given access to a fuzzer. I would guess that it probably did
| have access to a fuzzer to put together some of these
| vulnerabilities.
| gnarlouse wrote:
| A cybersecurity pandemic will surely be the Hiroshima that wakes
| people up to AI. /s
| kristofferR wrote:
| This is pretty insane. A model so powerful they felt that
| releasing it would create a netsec tsunami if released publicly.
| AGI isn't here yet, but we don't need to get there for massive
| societal effects. How long will they hold off, especially as
| competitors are getting closer to their releases of equally
| powerful models?
| charcircuit wrote:
| OpenAI did the same thing with GPT3 trying to scare people into
| thinking it would end the internet. OpenAI even reached out to
| someone who reproduced a weaker version of GPT3 and convinced
| him to change his mind about releasing it publicly due to how
| much "harm" it would cause.
|
| These claims of how much harm the models will cause is always
| overblown.
| imranahmedjak wrote:
| Building a neighborhood data platform that scores every US ZIP
| code using Census, FBI, and EPA data. Also running a job
| aggregator that fetches 37K+ jobs daily from 17 sources. Both
| free, both Node.js + Express.
| MisterBiggs wrote:
| What happens once an agent can reliably get 100% on swebench?
| zambelli wrote:
| I'm glad to see that it stands its ground more than other models
| - which is a genuinely useful trait for an assistant. Both on
| technical and emotional topics.
| caycep wrote:
| When do we get our Kuang Grade Mark Eleven icebreaker?
| meander_water wrote:
| I think this is a largely inflated PR stunt.
|
| Opus 4.6 was already capable of finding 0days and chaining
| together vulns to create exploits. See [0] and [1].
|
| [0] https://www.csoonline.com/article/4153288/vim-and-gnu-
| emacs-...
|
| [1] https://xbow.com/blog/top-1-how-xbow-did-it
| copypaper wrote:
| Yea, but can it secure systems from the unpatchable $5 wrench
| vulnerability?
|
| https://xkcd.com/538/
___________________________________________________________________
(page generated 2026-04-07 23:00 UTC)