OpenAI が Frontier AI の民主的ガバナンスに関する政策青写真を発表

Right after a new Executive Order seems like an excellent time to offer OpenAI’s new document: Democratic Governance of Frontier AI: A Blueprint For A Federal Framework.

OpenAI: We also see early signs of recursive self-improvement (RSI) in today’s systems: where AI development is itself accelerated by AI.

We expect this to increase competitive pressures among developers and nations, and create governance challenges that existing institutions are not equipped to address. As RSI emerges, societies will need ways to shape the trajectory of AI development and ensure that it serves human interests.

I choose the glass half full view of the above statement. Yes, this is not exactly leveling with you about the full scope of the problem, but at this point, I’ll take it.

OpenAI praises democracy, notes the United States is in a unique position, and calls for transparency and state capacity, especially the ability to evaluate new models, on the SB 53 model. They call for CAISI to be empowered, for good government, for maintaining our compute advantage and several other good ideas.

Implementation details matter a lot, but this document exceeds expectations a lot.

Peter Wildeford: OPENAI: "We also see early signs of recursive self-improvement in today's systems". RSI is "potentially the most consequential frontier safety issue of the coming decade."

Shakeel: NEW: OpenAI just released a blueprint for a federal frontier AI safety framework.

It calls for CAISI to perform a mandatory evaluation process of “the most capable frontier models” — but explicitly says “CAISI's role should be to conduct evaluations and recommend mitigations — not to approve or block deployments.”

It says companies should “implement appropriate safeguards” and that there should be “meaningful accountability mechanisms” if companies fail to comply with “safety obligations,” though it’s unclear about what exactly should happen if CAISI or internal evals suggest a model is too dangerous to deploy.

It also calls for a federal framework similar to SB 315, and for preemption of “state laws that seek to regulate the same frontier safety risks.”

Notably, the blueprint has a huge focus on recursive self-improvement, saying that CAISI should “treat RSI as an urgent priority” and develop standards for independent technical assessments that give “policymakers … ongoing visibility into progress toward RSI.”

Sam Altman's currently in DC, where he's meeting with policymakers including Speaker Mike Johnson and Minority Leader Hakeem Jeffries.

Dean W. Ball: This seems reasonable!

Having CAISI—a civilian agency—conduct this testing in primarily non-classified ways is the way to ensure it does not become a licensing regime. The Trump EO’s classification of the process raises the risk that testing morphs into a de facto mandatory permitting/licensing system.

The other document we got yesterday was a giant 269 page draft bill ominously called the ‘Great American Artificial Intelligence Act of 2026,’ after which I mostly saw a bunch of people complaining about those who had concerns about the bill’s preemption measures.

As always, I reserve judgment on this GAAIA Act proposal, until such time as I can RTFB (read the bill) and see what is in it. Details matter.

By contrast, except for tracking credibility, exactly who said what about who said what mostly does not matter. As we saw with SB 1047 people will just say things or claim other people just said things, a lot of those claims will be false or even absurd and there will be little or no consequences for those people.

Back to OpenAI’s blueprint.

What Do We Want?

They state these desirea:

Address frontier risks to national security and public safety.

Advance democratic governance.

Promote transparency.

Protect innovation.

Build adaptive institutions.

Those are great, and the full descriptions are better, these go beyond the basic applause lights although yes there are plenty of applause lights:

Address frontier risks to national security and public safety. The primary goal of any frontier safety framework should be to mitigate the most severe risks posed by advanced general-purpose AI systems. These include risks related to cyber and CBRN threats, RSI progress, and loss-of-control scenarios that could result in catastrophic outcomes.

Advance democratic governance. Decisions about how society manages frontier AI risks should be made through representative government, not by private companies acting alone. Frontier safety governance should reflect the strengths of free societies: transparency, public accountability, independent oversight, and the rule of law.

Promote transparency. Governments, researchers, businesses, and the public need reliable information about how frontier AI is developed, evaluated, and deployed. Transparency creates accountability, supports independent scrutiny, and helps ensure that policy decisions are informed by evidence.

Protect innovation. A frontier safety framework should focus on the highest-consequence risks without creating unnecessary barriers for startups, researchers, and developers building on top of frontier capabilities. It should reduce risk without locking today’s industry structure into law.

Build adaptive institutions. AI is advancing rapidly, and frontier governance must be capable of evolving alongside the technology. Policymakers should create institutions that can learn, experiment, incorporate new evidence, and update standards over time.

A common theme throughout the document is that OpenAI wants concentrate responsibility in CAISI, and avoid concentrating power in the NSA. I strongly agree with this and believe it is important.

These priorities are in conflict, but this is an excellent start.

The later parts of the document offer an implementation plan, which at the high level seems like clearly good ideas, that constitute a good start:

This blueprint outlines a three-part strategy for achieving those goals:

Building a national framework that leverages the emerging consensus reflected in state frontier safety laws.

Strengthening CAISI as the US federal government's primary institution for frontier AI safety.

Mobilizing a broader resilience plan across government to address the national security and public safety challenges posed by frontier AI.​

A National Framework

OpenAI proposes basing the national framework on California’s SB 53, New York’s RAISE Act and Illinois’s SB 315, as enforced by a strengthened CAISI:

Severe risk evaluations and mitigations.

Transparency requirements, including publishing frontier safety frameworks.

Independent assessment and auditing.

Critical safety incident detection and reporting.

Model weight security requirements.

Whistleblower protections.

Meaningful accountability mechanisms. Liability. No blanket safe harbors.

In exchange, Congress would preempt state laws around frontier safety.

In principle, if details are good, this could be The Way, the Minimum Viable Deal that someone trying not to die could consider accepting. There are five big concerns.

Meaningful accountability has to actually mean meaningful accountability, in addition to preserving liability for the underlying harms. Million dollar fines for violations do not cut it at this level. Billion dollar fines probably don’t, either, not on their own. We need to be able actually force compliance.

If we move these bills to the Federal level, then we all know who is in charge of implementation and enforcement. There is a real possibility that we pass a real bill with real consequences if enforced, and it just… isn’t. Or is enforced selectively and capriciously. Being able to only counts if you actually do it.

By default what happens is we take the watered down political compromises of the state bills, and then there are further compromises, watering them down further, when these are already pretty weaksauce bills. The initial offering risks being the best offer, the thing you then have to defend, rather than a first step. We need to go beyond the union of the bills, and beyond SB 315, before this becomes a deal worth taking, especially given the risks of non-enforcement. If we get into ‘this deal’s getting worse and worse all the time’ mode, it’s time to bail.

If we accept preemption of all ‘frontier risks’ this risks meaning that states can enact counterproductive laws about AI but not the ones that matter, and there is the risk Congress never again acts. We want to be very careful with exactly what we preempt. It’s fine and even good to rule out similar provisions (e.g. we’ve got the auditing and frameworks and transparency, specifically) but if it was all of ‘frontier risk’ and we despair of getting Congress to act again then that’s a very price to pay for modest transparency alone.

This could be treated as ‘we did it we fixed frontier risk’ and, well, very much no.

The basic problem is, if we agree to center all this enforcement at the Federal level, and accept preemption, then it needs to be enforced, and enforced with teeth, and the preemption properly tailored, and we have very good reason to worry this won’t be the case. And we have to worry that the deal we think we get doesn’t become one end of a negotiation or slippery slope. Once the preemption is in place, we don’t know that further modifications would be in the good direction.

Nathan Calvin similarly likes a lot of other things here, but is justifiably extremely wary of accepting preemption and giving up this reverse federalism. I agree that this part of the deal needs to be evaluated very carefully.

It’s all good to say ‘implement appropriate safeguards’ and ‘meaningful enforcement mechanisms’ but by default you end up with not those things.

Still, this is the serious proposal, or at least a serious starting point. I would seriously consider something built along similar lines.

Building State Capacity And CAISI

The first part is scary, because it is necessary but could easily backfire.

The second part is flat out necessary.

It starts out with properly authorizing and funding CAISI, giving them hiring authorities and public-service pathways and secure classified compute and a mandatory evaluation process that goes through CAISI rather than the NSA. Independent technical assessments should be supported.

Mandatory evaluation carries obvious dangers, but if it is time-boxed and does not on its own block deployment it seems hard to argue. As in, you need to get the evaluations, within a time frame that won’t appreciably slow you down. That’s something that actively helps frontier labs. It doesn’t mean you must block development if the numbers don’t come out right.

The tricky part is, if the answer comes back flashing red, what do you do? In practice, if the red flashes loud enough, this will cause the national security state and executive to act whether or not they are given specific authorization to do that, and also all the major labs will take the hint and cooperate. That’s the same logic of why the current ‘voluntary’ executive order is not all that voluntary.

Whole-Of-Government Resilience

This is a grab bag of other good things worth doing:

Facilitate collaboration and international coordination on frontier AI safety, especially communicating around progress towards RSI.

Protect America's compute advantage.

Restrict the adoption of unevaluated frontier AI systems. Yes, if you haven’t done the safety evaluations yet, don’t adopt the associated frontier models. Duh.

Ensure defensive capabilities scale faster than offensive capabilities.

Prepare for future resilience challenges

A bunch of this is rather vague, and it’s not a complete list, but yes, sure.

Reasonableness Rising

Thus, I am very happy to see OpenAI proposing this blueprint. This is a highly reasonable document, well exceeding expectations. If this becomes OpenAI’s position, including that of the PACs it supports, and they stop using various underhanded tactics especially false flags, then I would be fine with that.

We shall see. The discussions are going in good directions. My hope is to be in a position, soon, where it is hard to predict where my biggest concerns will lie with seriously proposed new bills.