「かつてありし未来の寓話」第5回：AnthropicがClaude Sonnet 5を公開

We, or at least ‘more than 100 American institutions,’ got Mythos back this week.

What we the people do not have is Fable or Sol.

While we wait for both Claude Fable 5 and GPT-5.6-Sol, today we instead got Claude Sonnet 5. As usual it will take a few days to get a handle on the new model. In this case, Anthropic is representing it as a cheaper and faster version of Opus 4.8, so even though the number says 5 this is a relatively minor development.

This post expands the Fable series to cover all further developments this week surrounding the Mythos Moment, and the various aspects of handling our new ad hoc licensing regime and figuring out policy going forward, and other aspects of policy as well.

This includes my notes on various rhetoric being pulled out, where I fear I end up saying similar things every so often, because we are doomed to repeat the cycle. I have accepted my role in that, but those are sections many of you can skip, and are marked in italics accordingly as per usual.

Table of Contents

You Should See The Other Guy. The other guy is the CCP.

DeepMind Coders Of The World, Unite. Or get back to work. Your call.

Report Your Incidents. Yes, you should probably do that.

Good Guy With An AI. You would very much like to stack the deck.

Free As In To Give It A Shot. The judiciary as AI regulator.

Everything Is Both Speech And Computer. They keep claiming this.

Lambs To The Slaughter. Goodbye, Humphrey’s Executor.

A Sign Saying Beware Of The Leopard. I never could get the hang of Thursdays.

The Once And Present Mythos. Ask if being in a top institution is right for you.

What Is To Be Done. Dean Ball offers his thoughts.

Distillation. Some people have views very different than my own.

What Would Banning Open Source Even Mean.

Open Weight Models Are Unsafe And Nothing Can Fix This.

You Should See The Other Guy

Important context for discussions these days:

Daniel Eth (AI Safety): “The U.S. places more restrictions on our frontier AI than China does on theirs.”

Yeah, but China places more restrictions on theirs than the U.S. did when our frontier was back where theirs is now.

Regulation is more needed when your model is more capable, or is closer to the frontier, and less needed otherwise. If you’re only producing Chinese-level results in America my understanding is you can do pretty much whatever you want. That doesn’t make our new restrictions good, but it puts it in perspective.

DeepMind Coders Of The World, Unite

Andreas Kirsch says that Google DeepMind and Demis Hassabis bet on a strong safety culture and good leadership built on trust, rather than on formal governance, and that when put to the test that failed. They still signed a Pentagon contract with enough weasel words that the Pentagon has all the leverage and can do whatever it wants. It does not matter what DeepMind’s people want, or that 600 employees signed a letter to not give in, because the government had too much leverage on Google, and the safety interests were abandoned.

The only way DeepMind’s employees would matter is if they were willing to use their leverage for real, and strike or quit. That didn’t happen this time.

Which is exactly why Kirsch and UTAW/CWU are fighting for Unite recognition, so they can fight for such things.

Report Your Incidents

Charlie Bullock highlights the new AI Incident Reporting Act, introduced by Rep Nate Moran (R-TX), which he says does a good job handling preemption, and uses a capabilities-based threshold for what is a covered model, which is tricky but in theory great.

Good Guy With An AI

The parallel with guns does not work the way you would like it to.

Sophia Cai and Ben Johansen (Politico): Essentially, he believes the best way to stop a bad guy with a dangerous AI model is making sure the good guys have access to the same model.​

The situation in which the ‘bad guy’ and ‘good guy’ have access to the same model is not the good scenario. That’s the bad scenario.

It is not as bad as the ‘bad guy has the superior model’ scenario. But the whole idea behind our policy is that the good guy needs access to an actively superior model, first, because if everyone has the same starting pistol then the bad guys can do a lot of damage before the good guys are done patching.

Part of this is that previously, the bad guys had a talent deficit, since most people do not want to be the bad guy. You had one black hat and a lot of white hats. But if the bad guy can scale as large as the funding, because the AI minimizes the need for talent, and doing bad things gets you more money, you have a rather large problem.

The open model paradigm can, at most, provide equality here.

There are claims that ‘at the limit’ the offense-defense balance favors defense, and superior tools will make everything more secure in practice, not less secure. I am skeptical of this, because of the ability to use automated tools to concentrate fire, and because I don’t think full bulletproof is a thing for most important places. But even if it is true, you still need to get to that point.

Realistic defense optimism depends on the defenders being given advantages and us working hard to make it happen. It is not an outcome that happens for free.

Free As In To Give It A Shot

So far AI policy has been the Executive throwing its weight around and Congress being unable to act. What about the judiciary? The judiciary can act with speed in some cases, and its rulings could do some very bizarre things.

For example, the First Amendment was not at all designed for this moment, but it could get weird fast.

Dean W. Ball: The most important legal questions in AI right now all relate to the First Amendment. What are the best fact patterns to demonstrate that the creation, distribution, and *use* of frontier AI is a form of protected expression?

Who, outside the labs, has standing to bring such suits? We need to move beyond ‘code is speech’ copium, and beyond the impulse to post into the void. Courts will be where the issues of the last two weeks ultimately get decided. It’s not going to be easy, given the national security implications, but also, the underlying technology is a large *language* model, and this should count for quite a bit indeed. The best legal minds of our time should be stewing over these and many related questions.

Some people in AI safety will respond, reasonably, with “won’t this risk creating a situation where AI regulation, including the friendly, softer kind you (Dean) support, is impossible?”

Realistically I think the answer here is no. This is part of why my regulatory focus has always been the frontier labs as *entities* rather than the models. It is true that 1A will place real limits on state intervention, and that some of those could bite in a world where e.g. it is desirable to regulate alignment in high levels of detail.

But this is the hard tradeoff of 1A, and it always has been. Sovereignty, in America, rests ultimately with the people. The First Amendment was, and is, a wild leap of faith. At some level, you have to choose: do we, the people, maintain our sovereignty, or do we not?

This does not mean ‘models should have free speech’ by the way

If we, the people - and yes, literally I mean any subset of humans, whether or not it is close to ‘those who are considered Americans today’ - want to maintain sovereignty in any meaningful sense, in the face of sufficiently advanced AI, that is going to mean a variety of controls on various aspects of such AIs. If we don’t do that, whatever else happens, such AIs will end up being the only ones potentially meaningfully sovereign.

In general and in practice in 2026, as a descriptive statement, I am more on the ‘not a suicide pact’ and ‘let him enforce it’ side of what I expect to be the judiciary’s role on this, where it counts, even if laws cannot be passed to override. Sufficiently crazy or unacceptable rulings in the AI space will get worked around, overridden or ignored.

Part of this is that I view a wide variety of things we already do, in non-AI contexts, as rather obviously unconstitutional, including a lot of ways we violate the First Amendment or claim to be ‘regulating interstate commerce’ and so on. That is both a descriptive statement of my view of the law as opposed to that of SCOTUS, and part of that is my normative support for much broader general free speech protections, and part of that is recognizing that these ships appear for now to have sailed, such that if the courts tried to enforce the actual words they would presumably fail.

Everything Is Both Speech And Computer

People are still determined to try. Thus, we are getting another round of people saying ‘code is speech.’ Technically yes of course code is speech, but a lot of things that we regulate these days are speech.

Some also now want to classify AI models themselves as speech and claim first amendment protections at the model level, such as Preston Byrne here.

I get where they are coming from, but there are lots of limits on things that are ‘similar levels of being speech.’ If we took everything that was ‘as much speech’ as an AI model, and said we could not restrict them, then our entire regulatory apparatus and civilization would work very differently.

Also, I do not think those advocating for the ‘freedom’ position understand the implications here, and are being insufficiently careful what they wish for, if this actually proved to have teeth, as opposed to ‘we control various levers of power over you so you will do what we say regardless.’

Preston Byrne: The user of an LLM is both speaker and listener. LLMs predict the next token in a context window; that context window is entirely user-generated, including both the user inputs and the previous LLM responses which would not have come into existence unless prompted by the user.

“Regulating AI” by controlling how people use LLMs is the regulation of expressive conduct. It might be the province of Congress to regulate other AI like self driving or industrial robots, but it does not likely have the power to control LLMs in the hands of ordinary people.

As I said above, I don’t think that holds as a practical matter of law, but courts surprise me reasonably often.

So suppose, in theory SCOTUS told the American government that once an LLM exists, they could not in any way control what was done with it, because any use of an LLM is expressive speech. And let us suppose, because of reasons, that there is no way around this in practice.

The obvious response of a government like ours - which to be clear I am not endorsing, I am only predicting, this is not normative - would be to prevent you from training a sufficiently advanced LLM, since this is too dangerous a thing to exist with zero controls on it.

Similarly, if once you get your hands on the LLM you can do what you want with it, then once the government has sufficiently woken up and the LLM is sufficiently capable, the battle would shift to not letting you get your hands on it unless you were someone it can control, or making you unable to run it in the first place.

The government is going to use the tools it has available. When you take away tools, it doesn’t drop the matter, it grabs whatever is left. The next issue is similar, if we can’t have independent regulators then we don’t get nothing. We get dependent regulators.

Lambs To The Slaughter

To that point, SCOTUS is if anything making this harder this week, not easier.

The 6-3 ruling in Slaughter v Trump overruling Humphrey’s Executor was not a surprise, but the quest for non-political regulations of all kinds got a lot harder this week.

SCOTUS has ruled that the President can fire, at any time, for any reason, anyone involved that he damn well pleases, with the exception of the Federal Reserve, which is, you know, because of reasons, the historical tradition and all that.

I think this is a rather terrible ruling in terms of its implications, and also on its merits, that will have far reaching consequences in other areas, and it also rules out what we would want to do to regulate AI without turning it into a political football or partisan weapon.

Ben Rossen: Trump v. Slaughter will have major implications for the future of AI regulation. If you want a federal body that can independently assess frontier models and then impose binding consequences - free from political or partisan influence -- that just got a lot harder, if not impossible.

Imagine Congress creates a Frontier AI Commission that can license major training runs, compel evaluations, restrict deployments, order emergency pauses, or impose penalties. Under today’s decision, its leaders could not be protected from presidential removal.

Why? The Court treated substantive rulemaking, investigations, enforcement, civil litigation, and in-house adjudication as executive power. A principal officer wielding those powers must be removable by the President at will.

… A general understanding that independence might be helpful is not itself enough: otherwise the same reasoning would protect the FTC, CFPB, NLRB, etc. The majority squarely rejected this technocratic argument.

Dean Ball disagrees with me and agrees with SCOTUS’s decision more generally, advocating ‘stripping away the fiction’ of nonpartisan agencies like FTC and SEC, and owning that they are partisan. In those cases especially, I strongly disagree. I think that even ‘the fiction of’ such non-partisanship is vital, and we should not return to a spoils system where the tools of financial and speech regulation are used openly for partisan advantage. Even if they’re going to somewhat be used this anyway, having to somewhat hide and be ashamed of, and not having the direct leverage, mitigates the damage.

How can we have a legitimate government, respected by the people, when it is known that such agencies are being used to hunt the President’s political opponents and favor his friends and those who pay him money? Do you think that, if a Democrat were to retake the White House, that this would not happen in reverse?

I used to think it was good to strip away hypocrisy in such spots. Now I have a better appreciation for Levels of Friction and what happens when you create common knowledge that the norms do not hold, and I no longer think that.

Dean Ball: Private governance institutions are one way to solve this problem. As I have written before: let’s vest political decisions--questions like “how safe is safe enough?”--in political institutions. But let’s place technical decisions--”once we’ve decided how safe is safe enough, how exactly do we determine this on a technical level”--in the hands of technical bodies overseen by government. A decision like Trump v. Slaughter isn’t the primary reason I was drawn to this notion, but knowing it would eventually happen certainly helped motivate my own research and advocacy of this idea.

In the case of AI, I don’t agree but I see practical reasons why this is reasonable, such as the need to pay dearly for expertise. But that completely doesn’t work for FTC or SEC?

Perhaps (mostly kidding, no don’t actually try to do this) we can move CAISI inside the Federal Reserve? I presume I am kidding, but also the main disaster scenario currently being contemplated is that cyber attacks compromise banks or the money supply. If we can place CAISI in Commerce, where it already clearly doesn’t really belong, why not the Fed?

Unfortunately, due to the Fed’s limited powers, we cannot move the actual AI regulator itself inside the Fed. We could in theory still move CAISI, if Congress wanted to, since it only provides information that others use to regulate, but it involves a host of other problems and also asking Congress to do it. I don’t actually want to do even this, because it risks turning CAISI’s focus into pure short term ‘what happens to the banks’ and looking away from AI R&D and even bio.

A Sign Saying Beware Of The Leopard

The basic situation these days:

MTS: SITUATION DETECTED: The Trump administration asked OpenAI to stagger the release of GPT-5.6 over national security concerns, per The Information.

Samuel Hammond: >pro X-ray glasses administration suddenly realizes X-ray glasses can see through their own walls too and begins to have doubts about marquee X-ray glasses promotion program.

A lot of people are suggesting instead that the leading X-ray glasses manufacturer is to blame because it kept pointing out X-ray glasses can also see through your walls, whereas the glasses should instead pretend they cannot do this.

Those glasses can, for example, be used to wipe out bank accounts, among other things, as he demonstrated to a Republican congressman.

Whereas Congress is on the level of ‘wait, the x-ray glasses are supposed to be AI?’

The new capabilities are scary, in large part because the people being scared did not even know about most of the old capabilities.

The Once And Present Mythos

The play by play:

Anthropic (June 26, 8:30pm): Since June 12, we’ve been working closely with the US government to restore access to Claude Mythos 5 and Fable 5. Today, the government notified us that Mythos 5, our strongest cybersecurity model, can be redeployed to a set of US organizations that operate and defend critical infrastructure.

We’re restoring access for these organizations quickly, and we’re continuing to work with the government to expand access to Mythos 5 and make Fable 5 available for general use again.

We do at least have Mythos 5 back, for ‘more than 100 US institutions, including major companies and government agencies’ as well as all Anthropic employees. Here is the front page of the letter Lutnick sent, without its Appendix A listing the institutions involved.

Fable 5 appears to be on track to return soon according to Axios, saying the Trump administration was as of Saturday close to restoring access. Success however depends on the NSA and Pentagon giving the green light, and relations with the Pentagon are Not Great, Bob.

I had not realized that throughout the whole Fable situation CAISI has been halted.

roon (OpenAI): the admin has pockets of technical excellence inside OSTP, and CAISI. pockets though. some really overworked guys, and I don’t know how empowered they are

Dean W. Ball: also worth noting that caisi has been on a stop-work order for most of this situation and ostp’s budget is comparable to the market value of a very-fine-but-far-from-top-tier home in the nicest neighborhoods of Washington DC. (Also rip collin burns.)

Does this all at least mean they care now?

Kinda. On some levels, yes. On other levels, no.

America takes a few narrow particular worries seriously now, but is definitely not thinking seriously about the problem of sufficiently advanced intelligence in general.

Dean W. Ball: While I have critiques of the export controls and other recent actions, there is a core truth that shouldn’t be overlooked: before the last few weeks, it was fair to question whether the United States took catastrophic AI risk seriously. Today, the answer is unambiguous: “yes.”

Eliezer Yudkowsky: Ahahahaha no. Unless by “catastrophic” you mean the kinds of dinky tiny cute catastrophes that have survivors.

Dean W. Ball: Idk if this is actually Eliezer but describing a pandemic as “cute” is psychopathic. I would suggest turning the reddit dial on this account down by a good 80%

Eliezer Yudkowsky: The only people I know taking even mere massive worldwide pandemics seriously are trying to scale private efforts to monitor wastewater for unfamiliar DNA. Where are the massive emergency mRNA vaccine factories, if some government is taking even lesser catastrophes seriously?

Dean W. Ball: when did “massive emergency mrna vaccine factories” become a)the barometer for taking ai risk seriously and b)a good idea

I mean, look, massive emergency mRNA vaccine factories are obviously a great idea,