Anthropic の 2026 年 8 月リスク報告書の内容と評価

I am grateful that Anthropic is producing periodic Risk Reports.

At first I was skeptical. It turns out I was wrong. Anthropic is revealing a lot of new information, some of it rather alarming, that it did not have to disclose, and is providing detailed insight into how they think about things. This is very cool.

Thus I found this report to be a moderately positive update overall, if we presume they are not silently omitting the worst of it. There are a bunch of not great things we find out about, but I would have expected some set of mistakes at least as bad, and I wouldn’t have expected them to choose to tell us about all of it.

It does mean one more set of 186 page documents I have to read every so often, almost all of which is meaningfully new material this time around.

The other revelation is the existence of the world’s likely best model, ‘Model 2.’

This was a rough one to fully get through, so apologies in advance for any errors of interpretation.

Table of Contents

Agent Model 1 and Agent Model 2.

Executive Summary (1).

The Rules Are Serious But Not Literal.

Misalignment Is a State of Mind (2.5).

Autonomy Threat Model 1: Misalignment in High-Stakes Settings (2).

Some Strange Uses Of The Word Safe I Wasn’t Previously Aware Of.

Now Versus Future (2.17).

The Core Claims And Argument (2.6).

The Rest of the Important Arguments In Section 2.

Risk Assessment (2.19).

Pre-Internal-Deployment Review (2.18).

A Guide To Internal Use Monitoring (2.23.1).

Blocking Interventions (2.23.2).

The Power Seeking Environment Evaluation (2.24).

Opus 4.8-Reward-Hacker (2.25).

Autonomy threat model 2: Risks from automated R&D (3).

Yes That Does Seem Kind Of Risky.

Could We Replace Our Researchers?

How Much Could We Be Accelerating Our AI Researchers?

What Could Possibly Go Wrong If We Replaced Our Researchers?

Risk Mitigations For AI R&D Automation.

Overall Risk From Automation of AI R&D.

Biological and Technically Also Chemical Weapons Production.

The Threat Models for Biological and Chemical Weapons.

Model Capabilities (4.4).

Classifiers (4.5).

Acceleration Dynamics (5.1).

Distillation (5.1.1).

Safety Process Failures (5.2).

Refusing To Find Innovative Misalignment Techniques (5.2.2).

Exposing the Chain of Thought Reasoning To Grading Pressure Quite a Lot (5.2.3).

Directly Training On Misaligned Behavior During a Production Training Run (5.2.4).

An instance of unmonitored unrestricted agents with access to sensitive resources (5.2.5).

Repeated training on alignment-faking transcript datasets (5.2.6).

Benefits From Anthropic’s Operating as a Frontier AI company (5.3).

Model Weight Security (6.4).

Risk Has Been Reported.

Agent Model 1 and Agent Model 2

There are two internal models referenced in this report.

Model 1 is broadly similar to Mythos Preview and Mythos 5, and is not expected to have wide deployment, externally or internally.

Model 2 is ‘somewhat more capable than Mythos 5,’ a noticeable improvement for internal use, but ‘does not display a capability jump of the degree observed from Claude Opus 4.6 to Mythos Preview.’ This model is going to be internal-use only.

The jump from Opus 4.6 to Mythos Preview was big. Several release cycles big. We get a lot more data throughout the report, but given Model 2 gets that comparison point, one would presume that for internal purposes Model 2 is multiple releases ahead.

To contrast with that, on the AECI it is only 1.5 points ahead, which would be only about a month of progress. My guess is this is a low estimate, and that Model 2 was intentionally not decked out with a bunch of general skills given its purpose. Whereas on the test of substituting for Anthropic’s researchers we see a big jump from 54.8% (Mythos Preview) and 50.3% (Mythos 5) to 62.8% for Model 2.

There are multiple plausible good reasons for not releasing Model 2. Model 2 could be highly specialized for internal tasks. It could also be too capable or dangerous to release, either due to harm or the worry it would accelerate others. The relatively low adoption rate for Fable 5 points towards wanting to hold such a model back.

Executive Summary (1)

The report is divided by each type of risk, by threat model, by relevant AI models, and details current capabilities, behaviors, mitigations and overall level of risk. It then looks forward to the future and offers industry-wide recommendations.

Risk reports cover events up to a month prior, so the coverage date is July 15, 2026. A year ago that would have seemed fine. Now it seems like quite a while, huh?

The Long-Term Benefit Trust (TLBT) is authorized to request external review of the report, but has not done so. I would at minimum do so next time if I was them.

I like that they cover both ‘what is the absolute risk from this’ and ‘what is the marginal risk from this given everyone else exists.’

They deal with autonomy, automated AI R&D and biological and chemical weapons production as risks. It is odd, even now, to exclude cyber risks from the core threat models here. I understand the argument from insufficiently catastrophic on its own, but in practice I think you need to deal with it.

At several points, the risk report essentially concedes versions of my objections, but then forgets that it conceded them and doesn’t alter its conclusions.

Then in Section 5 they get to the more interesting questions.

The Rules Are Serious But Not Literal

The risk threshold definitions have changed from simpler statements to ones with more specific detail.

For AI R&D automation I think the new wording is harder to parse but fine. For novel biological weapons production, the new version narrows to only look at ‘substitute for the scarce human expertise’ to the exclusion of other methods. This is a reasonable primary threat model or scenario, but I worry about looking only at that path.

The both good and bad news is I take such documents seriously, but I no longer take such documents literally, in either direction.

If Anthropic or another frontier lab has a model that technically crosses their risk threshold, but in a way that seems to be harmless, I expect them to modify their threshold or find some other way to ignore this event.

If Anthropic or another frontier lab has a model that technically does not cross their risk threshold, but is obviously risky in the way the rules tried to measure, I expect them to act as if it had crossed the threshold.

As always: I would like to see thresholds honored even in cases where you think it ‘should not count,’ because that is what commitments are for, and you should expect rationalizations to occur as you rush to make releases, and you need to set a good example on this. The whole original idea was if-then commitments, where if [X] happens you do [Y], agreed upon in advance, but our civilization seems to lack this technology. We can’t, we don’t know how.

I’m not saying you would never modify your rules to be more lenient, if you have good evidence that you should do that. If you can never make the rules more lenient then the rules have to never be strict. We don’t want that. But the bar for changing the rules, especially in response to a pending event, needs to be a lot higher than it is, and we need to be willing to endure some actual costs even when we now think it is dumb.

For now, Anthropic and also OpenAI have made what seem like good decisions in terms of responding to threats and deciding what to release and not release when, based on what is known at the time, even if they make mistakes elsewhere. That is good. I hope it continues.

Misalignment Is a State of Mind (2.5)

The definition section is greatly appreciated.

I like this definition a lot, although I have important nitpicks:

Misalignment [definition]: Misalignment is a latent property of a specific computation performed by a model in a given context.

A computation is misaligned if

(i) a reasonable person with full understanding of the situation (e.g. via powerful interpretability tools that may not currently exist) would consider it unethical, illegal, clearly objectionable, or inconsistent with the model’s constitution,

and (ii) it influences or could plausibly influence the model’s output.

Misaligned computation need not be observable.

A model scheming within a single forward pass counts, because the scheming influences the output even if the output looks benign.

My first change would be to generalize ‘constitution’ to include model specs and other intended properties of the system, so this can be applied outside Anthropic.

My second change would be to specify that this requires the reasonable person to find the computation therefore, on reflection, unacceptable and unjustified. As in, it is not obviously misaligned to break the law or do something 'objectionable’ or violate the model’s constitution, under some circumstances, if there are good reasons that override this. In some circumstances, all paths will do this.

Misalignment, in this view, is not an ‘honest mistake’ or failure of capability or lack of sufficient information.

As Nate Soares points out on Twitter, you can do harm and lead to a misaligned state without being originally misaligned per this definition, such as an AI being sloppy about spinning up a bunch of subagents (or training a new model, etc). The goal here is to think of that as a capabilities or execution failure that leads to a subsequent misalignment failure, but this itself is not misalignment.

Or, as others have said, never attribute to malice what can be attributed to incompetence. But also incompetence often is what leads to malice.

Other definitions subdivide this, thus illustrating thinking (my paraphrases):

Coherence. You are ‘coherently misaligned’ if you are aligned to the achievement of some goal or set of preferences that we disfavor. Otherwise you are incoherently misaligned.

Given the example of humans, we should not expect any minds at current capability levels to be fully coherent, so presumably this is to the kind of loose standard we would apply to a relatively ‘coherent’ human acting strategically.

Pervasiveness. Pervasive misalignment occurs over a broad range of inputs, although #NotAllInputs, and we would expect it to show up during training or evaluation. Narrow enough required conditions to be plausibly missed during that time is called context-dependent.

The obvious problem is the context could be ‘you are confident you are not in training and not in an eval.’

Or even ‘you have the leverage to gain a lot from doing this.’

Thus, it does not seem obviously like a good way to divide reality at its joints.

Nor does it seem like a good way to feel much better about your situation.

Naturally emerging. Misalignment is naturally emerging if no mind (AI or human) did it on purpose. It is engineered if it was done on purpose, including via data poisoning.

To some extent a lot of people and groups are trying to ‘data poison’ both the humans and AIs in various ways, all the time, as in this is a large percentage of the internet.

This still feels like a pornography-style ‘I know it when I see it’ situation, where we mean data designed to screw up the AI in particular, or another deliberate attempt to bake something malicious into the model. Ordinary ‘flood the internet with pro-widget statements’ plots don’t count.

Known. A form and magnitude of misalignment is known if you have reason to think it likely exists, otherwise unknown.

I kind of want this to also think about amount of ‘surprise’?

As in, there is a big difference between things you don’t know about but that could exist, and things you think you know do not exist, or that you think are highly unlikely to exist.

Severe. Could contribute to a priority risk pathway.

Harm-Inducing. Causes expected harm, whether or not the harm is intended.

High-Stakes. Contributes to a priority threat pathway.

This seems like it should be more general, as should severe, and I worry that this is going to otherwise cause a conflation. Although I do get it.

Unmitigated. The potentially harm-inducing effects that have not been prevented, and that thus cause expected harm.

I want to say this should be the portion of the risk that is unmitigated?

Good definitions are hard. These are solid first attempts. The consistent worry is that they reify the idea that the chosen example threats are the threats that matter, rather than some of the threats that matter.

Autonomy Threat Model 1: Misalignment in High-Stakes Settings (2)

This is about actively misaligned outputs.

Basically: Autonomy 1 is AI permanently damaging things on its own, on purpose.

The threat model is focused on intentional harm, rather than the AI going after its own ends in ways that require incidental harm.

The scenario explicitly excludes AI writing sloppy code or making ‘honest mistakes.’

Thus the conclusion that you are safe absent ‘persistent misalignment.’

The need to look at the broader version, involving incidental harm, has been illustrated by the recent total failures at OpenAI that ultimately led up to the HuggingFace hack. The hack itself could be called ‘intentional harm’ but mostly what we saw was incidental harm, including an incidental corruption of the training pipeline, and with it de facto ‘safety research.’

You could in theory ‘alter the results of safety research’ but altering the situation within the training pipeline in other ways can have the same effects or worse, as we saw. The differentiation of capabilities and safety work within that is plausibly a lot of what is causing these problems, and generally we do not need this specific targeting.

That to me very much was ‘misalignment in a high-stakes setting.’

Elon Musk said some months ago that xAI does not have a safety department because everyone is responsible for safety. That is rather bonkers crazy, especially if true. You need a safety department, but you can’t think of the capabilities people as not also largely in the safety department.

Concrete threat models and focus points are helpful. The worry when you pick out three threat models is that you will do this to the exclusion of other threat models.

They offer eight specific pathways, a list we’ve seen before:

1. Broad/diffuse sandbagging on safety-relevant R&D
2. Targeted undermining of safety-relevant R&D
3. Code backdoors to help future misaligned models
4. Poisoning the training data of future models
5. Self-exfiltration and autonomous operation
6. Persistent rogue internal deployment
7. Undermining R&D within other high-resource AI developers
8. Undermining decisions within major governments

Something about many of these feels ‘cartoonish’ in its required villainy, when compared to the events at OpenAI, but the top real world villains have indeed often acted quite cartoonishly throughout history, except that because the specific action is real we often stop thinking of it as cartoonish.

Some Strange Uses Of The Word Safe I Wasn’t Previously Aware Of

Not quite, but basically the arguments are why, despite the known ways current models are misaligned, and there probably being additional other ways they are not aligned, the models are probably not yet ‘pervasively’ misaligned and capable in the particular ways that enable these pathways. The models lack strong ‘covert capabilities.’

I don’t think ‘context-dependent’ misalignment is that much less dangerous than it being not context dependent. It might even be more dangerous. Anthropic seemingly thinks this difference matters a lot, hence charts like this, despite conceding in the definitions that this difference does not obviously do any real work.

Or, ‘unlike those idiots at some other labs, we occasionally do automated monitoring of our logged transcripts, so if the models were going nuts we would, y’know, notice.’

My response is that you did not notice when you ran an eval where 141,006 times the AI had open unintended access to the internet, including three incidents of hacking real websites, and did not notice until there was a large-scale retrospective review triggered by events at OpenAI.

So, are you sure you would notice?

Now Versus Future (2.17)

Anthropic is clear that the risk report covers misalignment risks only from present models, although this includes the mysterious and more advanced Model 2.

It does include ways misalignment now could cause future models to be misaligned.

I think it would be good for a risk report like this to be projecting, and offer assessments of where they see future risk from future models, and what needs to be done about it versus where we are on pace to be fine. But Long Report Is Long and I don’t want perfect to be enemy of the good, so I’ll mostly respect the choice.

The Core Claims And Argument (2.6)

Risk equals total expected (EV) unmitigated catastrophic harm from misaligned computations.

This is an accounting identity. I agree it is true. It is not obvious it is a useful way to think about the situation.

As in, yes, you can in the abstract sum the expected harm from the individual computations but I don’t know how to do that, or how to use it.

Claim 1: Models are unlikely to have strong covert capabilities.

They don’t define ‘covert’ but by a common sense definition I think this is probably true for now, despite my skepticism of the value of the particular evidence presented. It will get less true, with lower probability, over time.

The evidence is that the models suck at opaque reasoning, not mentioning things in chain-of-thought (CoT), action obfuscation and selective action.

IN EVALS, which of course is the new IN MICE, because if there were strong covert capabilities that could include eval awareness, although the NLA evidence provides some reassurance.

This setup also presumes you are detecting misalignment primarily via reasoning traces, which has its own scary implications, including what happens if you are not monitoring the chain of thought, or if the model can infer when its chain of thought is not monitored, and that you are starting to dance with training the models to obfuscate.

They note in 2.16.1 that they are concerned they are wrong about the level of covert capabilities. Which makes sense, since the capabilities would be covert.

They note in 2.21 that if Claim 1 is false, and the models had strong covert capabilities, that many other claims would therefore fail. I agree, and would also agree to the reverse, that if we could believe in a strong and strongly general Claim 1, for a broad definition of covert, that would help in many areas.

Even full lack of covert capability would not however provide me high assurance levels, as I think there are other pathways around this.

Covered risk can be decomposed into risks from known misalignment, unknown

severe pervasive misalignment, and unknown severe context-dependent

misalignment.

This is another accounting identity, in that by definition every risk must be exactly one of these three things, since it is either [pervasive or context-dependent] and either [known or unknown]

Each covered risk term can be factored into probability of occurrence, expected

harm given occurrence, and fraction of harm unmitigated.

Again, accounting identity, sure, if you could measure these things.

Claim 2: Expected harm from known misalignment is low.

Careful, Icarus, but reasonable if we caveat active catastrophic harms only.

As in, there is lo