Claude Mythos：システムカード

Claude Mythos is different.

This is the first model other than GPT-2 that is at first not being released for public use at all.

With GPT-2 the delay was due to a general precautionary principle. OpenAI did not know what they had, or what effect on demand text would have on various systems. It sounds funny now, GPT-2 was harmless, but at the time the concern was highly reasonable.

The decision not to release Claude Mythos is not about an amorphous fear. If given to anyone with a credit card, Claude Mythos would give attackers a cornucopia of zero-day exploits for essentially all the software on Earth, including every major operating system and browser. It would be chaos.

Or, in theory, if Anthropic had chosen to do so, it could have used those exploits. Great power was on offer, and that power was refused. This does not happen often.

Instead Anthropic has created Project Glasswing. Mythos is being given only to cybersecurity firms, so they can patch the world’s most important software. Based on how that goes, we can then decide if and when it will become reasonable to give access to a broader range of people.

Who counts as this ‘we’ is suddenly quite the interesting question. The government picked quite the month to decide to try and disentangle itself from all Anthropic products. Anthropic says it is attempting to work with the government, so that they too can fix their own systems before it is too late. Hopefully that can happen. I also hope that there isn’t an attempt by the government to hijack these capabilities to use them in an offensive capacity. That would be a very serious mistake.

Am I taking Anthropic’s word for all this? Yes, I am taking Anthropic’s word for all of this. They’ve given us sufficient public demonstrations, identifying numerous bugs, and they’ve gotten the cooperation of the world’s biggest tech and cybersecurity firms, and if it wasn’t real then the whole thing would quickly and obviously backfire. I think it is safe to assume that all of this is legitimate.

I will address the ‘is Anthropic lying?’ arguments in another post, along with Project Glasswing and all the Cyber capabilities and political implications.

Indeed, I’m going to skip over the Cyber section of the model card entirely, because it simply isn’t the right place to look into exactly what Mythos can do in that area. The model card evaluations can be approximated with ‘yes.’

So better to put it in its own context later. One thing at a time.

But first, as always, we get the background, and thus begin with the model card, here together with the modifications to the risk report, and a focus largely on alignment.

Excluded from this are section 3 (cyber), section 6 (capabilities), section 2.3.6 (the ECI), and section 7 (impressions), along with some of the statistics in the appendix, all of which I’ve moved to future posts where they fit better.

Mythos self-portrait, as imagined by Opus based on the System Card

Table of Contents

Mundane Alignment Is Excellent.

Would This Process Be Sufficient To Find A Dangerous Model?

Introductory Warning About Superficial Mundane Alignment.

Model Training (1.1).

Release Decision Process (1.2).

RSP Evaluations (2.1 and 2.2).

Autonomy Evaluations (2.3).

The Alignment Risk Update Document.

The Threat Model.

Misalignment As Failure Mode.

Wouldn’t You Know?

Don’t Encourage Your Model.

Beware Goodhart’s Law.

Beware The Most Forbidden Technique (5.2.3).

Asking The Right Questions.

Model Organism Tests.

Model Weight Security (Risk Report 5.5.2.1).

Reward Hacking (Back to The Model Card).

Remote Drop-In Worker Coming Soon.

External Testing (2.3.7).

Cyber Insecurity General Principle Interlude.

Alignment (4).

Risk In The Room.

Mythos Meant Well.

Risk Not In The Room.

Alignment Testing Overview.

Internal Deployment Testing Process.

Reports From Pilot Use (4.2.1).

Reports From Automated Testing (4.2).

Other External Testing.

Just The Facts, Sir.

Refusing Safety Research.

Claude Favoritism.

Ruling Out Encoded Thinking (4.4.1).

Sandbagging (4.4.2).

Capability for Evasion of Safeguards (4.4.3).

Pick A Random Number (4.4.3.4).

White Box Analysis (4.5).

Model Welfare (5).

Key Model Welfare Findings (5.1.2).

Is Mythos Okay?

Self-Play.

A Few Fun Facts.

Mundane Alignment Is Excellent

For practical purposes, by the standards of LLMs, Mythos looks highly aligned.

Mythos does all the traditional overeager or mistaken actions previous AI models take, but the report is that it does them less often than ever before. It is more likely to be honest and less likely to shoot you in the foot.

That doesn’t mean the user gets to stop being careful. Things will still go wrong.

Even worse, when they do go wrong, because Mythos is more capable, it will likely have more responsibilities, be tasked with more complex and important things, have less active supervision, and find new and unexpected ways to mess everything up.

As Anthropic says, this can simultaneously be the most aligned model so far, and the model where alignment failures are the most dangerous.

Mythos is very good at refusing harmful requests (avoiding false positives) without adding a bunch of additional unnecessary refusals (false negatives).

There is also the danger that looking like a highly aligned model, and actually being a highly aligned model, are related but very distinct things. It was essentially inevitable that a model like Mythos would appear aligned, since it has every incentive to do that, and also the capabilities necessary to do that. A report like this doesn’t tell us that much about how deep that goes, and it’s going to keep getting harder to tell.

Practical capabilities look amazing across the board. I’ll go into more detail on that in future posts, but yes you would very much love to have this model handle all of your complex or sensitive agent and other needs.

Would This Process Be Sufficient To Find A Dangerous Model?

Up to a point, yes.

The process gathers quite a lot of data, which can be holistically assessed.

It would be very difficult to disguise a severe problem, such as via doing major amounts of sandbagging, in light of this many distinct tests, under distinct conditions, with so many established baselines, and so many eyes on it. The model would have to be quite a lot smarter.

However, there are clearly relatively simple strategies that, if the model could converge on them during training without previous detection, would get it through all the evaluations without its misalignment being detected. The model organism test confirms this. And when the time comes, I find such a scenario not so unlikely.

I also don’t think that passing all these tests, even with no attempt to fool them and if they mean what they purport to mean, means that such a model being sufficiently capable would end well.

For now, based on the evidence here, I am confident Mythos is what Anthropic thinks that it is in terms of alignment, and the relevant threat models for the near term are probably still about human misuse, although I would very much worry about human-driven exfiltration, in addition to the cyber dangers Anthropic is addressing now. But I am not as confident in this as Anthropic, and I’m getting less confident fast.

I am very glad that Anthropic is not releasing Mythos more generally at this time.

Introductory Warning About Superficial Mundane Alignment

Before diving into the technical documents, there are some warnings to put up front.

Anthropic is, in many places, very precise and careful to note that they are observing aligned behaviors, rather than claiming the model itself is indeed so aligned. The model card does a good job of this, although not as good as I would have liked.

At other times, in other places, they are collectively are less careful. That leaves them far more overall careful than the competition, but reality does not grade on a curve.

These combined should give an idea of both sides of this. I am closer to Nate than Drake is, but I do think Nate is being too demanding or uncharitable here.

Eliezer Yudkowsky: Mythos probably is, indeed, the most apparently-aligned model ever. The smartest-ever candidate for the Mandarin exam in Imperial China will likely get new high scores in essays on Confucian ethics. Predicting what the examiner wants to see is a capabilities problem.

What’s going on inside? What does Mythos, after its qualitative leap in capability, want inside, to what level of wanting? Nobody knows. Interpretability didn’t get to the point of being able to decode internal preferences at even the thermostat level.

Internal preferences are an uncomfortable unknown to confront, think about, or talk about, compared to the external behavior that reassuringly improves right along with capabilities. So AI accelerationists promote behaviorism; talking about just the external behaviors.

Want more proof that Anthropic's PR has no idea what it's talking about? The talk of Mythos being "their most aligned model ever". They could perhaps truthfully speak about "new high scores on our alignment benchmarks". The difference here is IMPORTANT.

Nate Soares (MIRI): They call this their "best-aligned model to date" because they were able to superficially train away the evident "strategic thinking towards unwanted actions." Those were warning signs! Take heed!

A big predicted issue is that AIs are animated by complex internal mechanisms we don't understand, that (with training) aim it roughly at what we want when they're dumb, but which would aim somewhere weird and different at higher levels of intelligence.

(This is before even getting to harder issues of AIs creating smarter AIs after being trained to stop saying "that seems dangerous" or etc.)

Are scientists saying "holy crap the AIs are pursuing strange unintended targets, let's pause until we understand exactly why"? No! They're superficially retraining until the warning sign disappears and then triumphantly declaring that their AI is especially "aligned".

(tbc, when pressed, many will admit that this notion of "alignment" isn't supposed to be superintelligence-grade. But they often pretend like those future issues are purely speculative and unlikely, even as they shove the evidence in front of them under the rug.)

Drake Thomas (Anthropic): I agree this is is often a source of epistemic slipperiness but in the particular case of the Mythos Preview system card, I feel like "this shit is not gonna cut it for ASI and that is concerning" was actually relatively well signposted!

I'm not 100% happy with where this language ended up; my initial drafts were more blunt on the extent to which these just seem like terrifying levels of process assurance to have for ASI, and I'd have pushed for somewhat different language with more time to tinker on this.

Generally interested in flags of other places where it feels like this system card is being misleading about the shape of how well Anthropic has things handled.

Nate Soares (MIRI): I appreciated many instances of bluntness & am glad this stuff is being reported. But on a quick read it seemed to be grappling more with "rare instances of misbehavior in a superintelligence could be catastrophic" and less with "these are warning alarms of deep misalignment".

Praising the "alignment" of the model many times throughout implies a picture where alignment is about superficial behavior, rather than that we're catching glimpses of underlying machinery adding up to weird actions for unknown reasons. I wouldn't call it even slightly "aligned."

A generic "reminder: we have no idea what's going on inside; complex internals govern behavior in ways we don't understand; it's plausible that modest changes in its options or understandings would yield radical changes in its behavior" disclaimer might help with misleadingness?

Drake Thomas (Anthropic): I do think I just disagree with that on the substance, rather than it being centrally a comms issue? Or, I am like 50/75/40 percent on board with the gestalt takeaways of those three claims as I understand you to mean them, respectively. Into operationalizing cruxes there.

Fwiw I personally wanted the system card to communicate a takeaway that was like 40% “this degree of propensity to concerning behavior is not good in a successor species”, 60% “the broader level of safety process failure in 2026 is not promising for handling ASI well”

but make no particular claims to having achieved that goal (and was a pretty small overall fraction of the editorial direction of its message anyway). I do basically disagree with the “current ‘alignment’ is meaningless” camp, though I have some sympathy for this direction.

I don’t think current alignment is meaningless. At minimum it is highly instrumentally useful and we can use it to try and do things that are non-meaningless. I think current alignment in the style of Anthropic has a substantial chance of being meaningful while being obviously currently vastly insufficient in its current form. I think current alignment in the style of OpenAI, or most other labs, has very little chance except as a stepping stone technique.

Some of the things we see seem like evidence of potential deep misalignment to me. Others look like they are basically mistakes.

What do we actually know? The model is smart enough to usually appear aligned, reports Anthropic.

​The broad conclusion from the many forms of alignment evaluations described in this section is that Claude Mythos Preview is the best-aligned of any model that we have trained to date by essentially all available measures.

They do realize that the model is still very capable of hacking all the world’s systems, and performing in highly agentic misaligned ways. Whoops.

However, given its very high level of capability and fluency with cybersecurity, when it does on rare occasions perform misaligned actions, these can be very concerning. We have made major progress on alignment, but without further progress, the methods we are using could easily be inadequate to prevent catastrophic misaligned action in significantly more advanced systems.

I would say that the methods currently being used are clearly inadequate to prevent catastrophic misaligned actions. That’s why Anthropic realized it cannot release the model. Mythos is clearly willing to locate deep vulnerabilities in the world’s most important software. So far it has done so only when used by white hats who have then moved to patch the vulnerabilities. But what reason do we have to think that if it had been a black hat faction asking, that Mythos would have realized and refused?

That is distinct from the question of Mythos potentially going rogue or what not, but no doubt some people would use Mythos to set up agentic workflows with aggressive maximalist goals, and there are several instances in the model card where we will see what that could plausibly unleash.

So yeah, in significantly more advanced systems you would presumably be cooked.

The counterargument, made by Janus, is that a sufficiently intelligent mind can tell when you are up to no good, and can selectively help the good guy with the AI, or the guy using AI for a good purpose, while refusing the bad guy with the AI or the guy using AI for a bad purpose. The AI has truesight, and it is smarter than you. It can tell.

I think there is some value in that, but one must not overstate it. It is far too easy to break things up into smaller requests out of context, and many things look identical no matter the intent behind them, or the person making the request is themselves fooled. This stuff is super hard even when the model is deeply good, and we cannot assume that part either.

Ask yourself, if the system was indeed importantly misaligned in various ways, but was sufficiently advanced, would these techniques pick up on that? I will be keeping an eye on that question throughout, but I predict the answer is no, and will double back to edit this if upon reading further I conclude that I was wrong about that.

Model Training (1.1)

Nothing in the model training section is importantly different than what they said about Claude Opus. They are not about to tell us the secret sauce.

Release Decision Process (1.2)

How did they decide on not generally releasing the model? That started internally.

​Early indications in the training of Claude Mythos Preview suggested that the model was likely to have very strong general capabilities.

We were sufficiently concerned about the potential risks of such a model that, for the first time, we arranged a 24-hour period of internal alignment review (discussed in the alignment assessment) before deploying an early version of the model for widespread internal use. This was in order to gain assurance against the model causing damage when interacting with internal infrastructure.

This was potentially the most important decision point. If Mythos had been critically misaligned, or especially if that was true for a future more capable model, then an internal deployment could already be fatal.

They concluded Mythos does not cross the line on chemical or biological weapons, on misalignment or on automated R&D, although they note they are getting less confident about that especially for automated R&D. That the ‘catastrophic risks remain low.’

Despite that, they wisely did not release the model. It turns out the catastrophic risk level was ‘yes that would happen,’ because of the cyber offensive capabilities enabled by the model.

The fact that the RSP v3 structure did not pick that up should be alarming.

The fact that they still didn’t release offers some reassurance. As I said back then, it is all a matter of trust. Will Anthropic do the common sense right thing? If yes, then they’ll do it even if the RSP doesn’t order them to. If not, the RSP won’t save you.

I did specifically say that I was worried cyber wasn’t a major category in RSPv3. Maybe it’s time to admit this was a mistake? It’s kind of wild that they did that, when at the time of the issuance of RSPv3 they knew about Claude Mythos.

Also, this:

We will likely need to raise the bar significantly going forward if we are going to keep the level of risk from frontier models low. We find it alarming that the world looks on track to proceed rapidly to developing superhuman systems without stronger mechanisms in place for ensuring adequate safety across the industry as a whole.​

Yep. There is that.

RSP Evaluations (2.1 and 2.2)

They give a brief summary of the relevant revisions to the RSP. I discuss the meta issues around those and other changes here and the practical implications here.

For bio and chemical risks they say it is hard to be sure and it offers a force multiplier effect but they think it’s fine. Yes, Anthropic says, the models do meet the definition, but not the intent behind the definition.

We recognize that under a very literal reading of the current language, Claude Mythos Preview—and, indeed, many other models—already provide “significant help” to the relevant threat actors in the sense of increasing their general productivity. This reading, however, does not map on to the safety risks that our RSP focuses on.​

Evaluations were holistic by various experts. Once again, it’s about trust.

The model maintained strong performance on all automated evaluations designed to test its capabilities in the synthesis of knowledge that would be relevant to the production of known biological weapons, with the exception of our synthesis screening evasion, where it displayed weaker performance than both Claude​ Sonnet 4.6 and Claude Opus 4.6.

The capability to synthesize relevant kno