AI ガバナンスの AGI 時代へ：米国行政機関が Anthropic の最新モデルへのアクセスを強制停止

The executive branch of the United States forcing Anthropic to turn off access — both internally and externally — to their latest Claude 5 Mythos/Fable models is the starting gun of a new era in AI governance. This is the era defining AI agents that effectively complement human workers, unlocking new ways of working and new domains of applying existing tools. The models will keep getter better at a rapid pace, forcing more governance challenges.

Share

An unstable equilibrium at the frontier

On Friday, just after markets closed, the government reached out to Anthropic to force them to suspend access to their model to any foreign national or user abroad. When writing this, the saga is still unfolding. It is likely that Anthropic and the Government reach an agreement to re-release the model, but it is a messy indicator of new types of AI governance. The White House was tipped off to the risk by Anthropic’s largest financial and technology partner, Amazon.

These dynamics are very complex, and while the picture is incomplete I will only offer commentary on what will likely be a lasting opinion of the saga:

An export ban on any model weights is going to be a lasting, negative policy for the U.S. This goes for both open and closed models, despite the open models being more likely to get import banned soon.

There was a legitimate cybersecurity concern regarding a jailbreak of the Fable model, even if it was very narrow. I do not personally think this should’ve set off this series of events, as no model is perfectly immune to jailbreaks and we need to prepare our infrastructure for a world where every global entity will have access to models of this capability level in the coming years.

Anthropic’s constant fear-mongering over the last few years has accelerated this moment. Without the constant messaging comparing AI to nuclear weapons, etc., I suspect this style of governance would not have manifested for another 6 to 12 months (of AI progress). There is some amount of Anthropic reaping what it has sowed.

People should not be taking a victory lap at our nation’s leading AI company being repeatedly subject to potentially political attacks (yes, I’m looking at people in the open-source community). This is economically highly unstable and undermines the economic stability of the American system. Sufficient instability could cause an economic recession and pop the AI bubble.

The White House’s actions to ban the model are heavy handed and are influenced by a strong political inclination against Anthropic. There’s a deep contradiction in the government’s demands and goals, as there is no domestic AI industry if foreign nationals cannot build with frontier AI in the U.S.

It is unclear to me as to why Amazon needed to take the information they had directly to the White House, what the lead-up to this release and ban were (lots of accounts online of pre-release discussions), and how crisis management between leading technology companies and the White House would normally play out. This points to weird dynamics with how Anthropic interacts with the government, which could be a reaction to being politically singled out.

Whatever the cause, it is a bad dynamic and casts a shadow over how Anthropic engages here. If a private company tries to control the government the government will push back stronger. I do not think Dario was actually at a wellness retreat (as said by a White House representative) but I do think he took longer to answer the call than is acceptable with who was on the other end of the line.

It’s hard for me to write anything cogent about these back and forth dynamics but they are very important to the truth of what is happening.

This picture is very complex. It points to a near-term world where model releases are judged on vibes by an executive branch with minimal technical talent, gating releases behind a blur of politically judged technical assessments.

This is a government that has internalized that we are in the AGI era. They were not ready for it and feel like they must act fast to regain lost time.

This is a government that took office when we were still in the ChatGPT era of AI governance — models that just answer questions. Their original AI Action Plan reflected this era, highlighting how a lot of previous safety discussions were more smoke and mirrors than concern, and how badly we needed open-source models to catch up. On balance, their opinions and prescriptions here were mostly agreeable and useful for the industry.

It is important to remember that even in this ChatGPT era of governance, the AI companies often told us that the models were a risk and could not be released without great care. Many spokespeople at leading labs didn’t give themselves enough headroom in language escalation for when the models took a meaningful jump in performance.

As we shift from answer inference to agentic inference, I’m still on the side that the risks have been exaggerated, but the margin between the real risks of the latest AI systems and the language used to describe them has shrunk. This has spooked power structures outside of the leading AI labs.

Entering the AGI era of AI governance is when a lot more sovereigns around the world start to take AI seriously. Europe, the Middle East, and likely also China are realizing they could be left without frontier AI. This swing is something that the open-source community takes as a win, seeing the long-term trajectory where businesses will want to control their own intelligence when the government can arbitrarily rule against the leading platforms.

The key to understanding this era is that these events between Anthropic and the federal government, which seem like two of the biggest events in the history of AI policy, are just the starting gun to what is to come. This is the new normal, as existing power structures feel the need to assert their control over the rapidly rising technology. This will be messy, fraught, and at times even perilous.

When following these events I’m filled with a sense of dread, staring into the void, because I know more is to come. It’ll take resolve to follow de-escalatory paths that help us achieve the positive visions of broadly diffused, safe, and cheap super-powerful AI around the world.

Subscribe now

Sovereign AI, open-source needs, and battles soon to come

The open-source AI advocates wildly celebrating this sequence of events aren’t remotely ready for when the spotlight of rapid-response AI policy turns its gaze upon them. It is very likely that a similarly aggressive action against an open model will come, but we don’t know if this is in 3 months or 2 years.

Read more