フロンティア研究所従業員がペース配分を要求する公開書簡に OpenAI と Anthropic が賛同

The most important open letter in years dropped yesterday.

This letter noticeably increases my hope that we will manage to not die, and that we will otherwise be able to secure for ourselves a positive future, both by its impact and by the evidence it provides that such a letter can get this level of support.

Signed by 1,224 employees of frontier labs including many heavy hitters, and now endorsed by both OpenAI and Anthropic, here is its full text, which I also endorse:

AI could help create a dramatically better future, but that outcome is not guaranteed. The world's leading AI companies believe they could be close to automating AI research. It is hard to predict exactly how much this will accelerate AI progress, but there is a real risk that capability development rapidly accelerates beyond our ability to understand or control the resulting systems.

To realize AI's potential, industry, government, and society at large may need the option to buy time to address emerging risks, develop security measures, and strengthen oversight. But each company—and country—is under intense competitive pressure not to unilaterally slow that acceleration. And today, the world lacks the technical and governance tools to deliberately pace frontier-wide progress.

Building on work already underway to monitor frontier model releases:

We request that the U.S. government support an international effort to develop the technical and governance tools needed to deliberately pace the frontier of automated AI development.”

• 1,224 employees of frontier AI companies​

The ask is to prepare mechanisms for future coordination, not a call to change the pace of development now.

I strongly endorse that ask, but even if you disagree with the ask, this shows one thing unmistakably: Many employees of the major labs expect automated AI research to accelerate soon, and they are terrified of what might happen next and how fast things will go. We are not prepared.

No matter what else you take away from this, believe that part.

Table of Contents

A Very Good Letter.

Who Signed The Letter.

We Need To Prepare Now So We Have The Option To Do This.

Words From Some Of Those Who Signed.

Words From Others.

A Good Start.

What The Letter Does Not Say.

What Happens Now?

A Very Good Letter

I would go somewhat farther than the letter does, especially in terms of explicitness about the existential risks that we face, but this letter is:

A fully correct position on the merits.

A masterfully worded and executed letter.

On the Pareto Frontier of what a letter can say versus who would then sign it.

An important step towards taking real efforts to make us all not die.

Strong evidence that support for us all not dying is stronger than we thought.

There were several key moves involved, especially:

Drawing the distinction between the action of laying groundwork for future intervention, versus calling for that intervention to take place right now.

Drawing the distinction between pacing development, versus previous terms like pause, slowdown or shutdown.

Not explicitly calling out the full nature or scope of the existential threats we face.

This allows people to sign on so long as they support the necessary current move of getting ready to act, even if they don’t think there is enough evidence yet to pull the trigger, or it is not yet time, and without having to endorse a particular view of the risks we face. The situation is rather overdetermined, so one can and should support this letter’s statement and ask even if you are doubtful of stronger claims.

Naturally, the first move of many opponents is to ignore those distinctions, whether or not they even noticed them in the first place.

A variety of beliefs are reasonable about whether intervention is needed imminently in order to pace development, and if so what degree of intervention. This was true before we learned of an internal OpenAI model hacking HuggingFace, which should update your perspective. It remains true now.

Whereas I believe the letter’s statement would ideally be uncontroversial. We do not know what the future will bring. In many of those futures, we will need the ability to steer outcomes, including via pacing development. This will require cooperation, between companies and ultimately between governments. If we do not lay the groundwork in advance, it will be too late, and we either will be unable to coordinate or will have to do so via much worse mechanisms.

Who Signed The Letter

OpenAI endorses this letter.

OpenAI: At the core of our mission is working through how to ensure increasingly powerful AI benefits everyone.

We believe that, at some point in the future, AI acceleration for frontier model development may be so high that the world will need to pace the rate of AI advancement.

We hope to contribute to work led by the U.S. government, alongside other labs and the open-source community, to develop the tools and mechanisms that could make that possible.

Key figures from OpenAI that have signed include Chief Scientist Jakub Pachocki, Chief Research Officer Mark Chen, Cofounder and Head of AI Resilience at the OpenAI Foundation Wojciech Zaremba, Roon, Head of Strategic Futures Dean Ball, Head of Safety Saachi Jain, and former Head of Mission Alignment Joshua Achiam.

Jake Halloran: Roon signing the letter as roon goes kinda hard

roon (OpenAI): honestly I signed it as my real name but PTF people decided it’s cooler if I’m roon

Anthropic endorses the letter.

Anthropic: We support this petition, signed by our CEO, several co-founders, and senior staff. Our own research on recursive self-improvement, published last month, points to the need for tools to deliberately pace the frontier of AI development so society can prepare. We’re glad to see broad agreement across the field.

Key figures from Anthropic that have signed include CEO Dario Amodei, co-founder Jack Clark, co-founder and Chief Science Officer Jared Kaplan, Co-Founder Benjamin Mann and Co-Founder, Interpretability Research Lead Chris Olah, Jan Leike, Ethan Perez and Claude Code creator Boris Cherney.

Key figures from Google DeepMind that have signed include Chief Strategy Officer Jasjeet Sekhon and VP of AI Safety & Alignment Anca Dragan.

Other key figures include Meta Chief Scientist Shengjia Zhao, Meta VP of AI Research Dawn Song, Mta Director of Alignment and Risk Summer Yue, Thinking Machines Chief Scientist John Schulman and Inherent Chief Scientist Edward Hughes.

A little under half of signatures are from Anthropic as of shortly after the announcement (as of writing this there are now 90 more signatures), with a lot of signatures from OpenAI and DeepMind, and a substantial number from Meta. The only one conspicuously missing here is SpaceX (xAI), or what is left of it.

Or by percentage of employee pool as of the evening of the 28th, notice that OpenAI now has more employees than DeepMind does:

⿻ Andrew Trask: Signature Statistics:

• Anthropic: 546 / 5,567 = 9.8%
• OpenAI: 350 / 10,473 = 3.3%
• Google / DeepMind: 199 / 10,219 = 1.9%

Total: 1095 / 26,259 = 4%

Denominators come from LinkedIn July 28, 2026. Numerators come from "CMD-F" from the website.

Efforts seem to have concentrated on the higher end of the employee pools. A lot more than 4% of the biggest names signed.

We Need To Prepare Now So We Have The Option To Do This

I have been emphasizing this distinction for a while.

Do I think we should deliberately slow or even pause the pace of AI development at this time, given the tools we currently have available to do that?

Until the hack of HuggingFace, I would have said:

No to forcing labs or everyone to do this for the sake of doing it.

Our tools for doing this are not good, and also it would be premature.

Yes to shifting resources into alignment and safety work.

I think this would be selfishly good for the individual labs, and that they are all massively underinvesting here. We are not on the Pareto frontier.

If this slowed down ‘raw capabilities’ progress a bit I think that would be totally fine. I would expect greater AI diffusion and sales as a result. Win-win.

You could take this too far, but in practice no one is going to do that.

Yes to laying the foundation for future interventions and coordination around those interventions.

The time may come, potentially quite soon, when this is no longer premature. If that day comes, we won’t have time. We need to get ready now.

This includes building transparency and state capacity, and laying groundwork for inter-company and international cooperation.

Yes to requiring better transparency, and better testing and safety protocols, and for requiring individual labs to pause frontier model training and other capabilities work if they cannot pass such requirements, until such time as they can pass.

After the hack of HuggingFace and other recent disclosures, it looks like OpenAI in particular may need to halt and catch fire to solve some severe misalignment problems, as Sam Altman said in a recent podcast they have at least partially done, and this reinforces the need for Anthropic and others to check if they need to do likewise. We are at or nearing a critical point.

This makes laying the groundwork for well-executed coordinated action that much more important and urgent. We are running out of time. Capabilities progress is accelerating.

We won’t be able to do the good version of this without solving a lot of very hard problems. The more work we do in advance, the better our option set becomes. This includes diplomatic work, technical work, governance work, the building of state capacity, increases in transparency, developing good corporate infrastructures and procedures, and also coordination work.

Words From Some Of Those Who Signed

I turn the floor over to statements from some of the signatories. There are many more good quotes available on the letter’s website.

Dean Ball (OpenAI): I am proud to have signed the letter linked below. I do not know whether or when it may be necessary to deliberately lower the rate of AI development, but I do know that we need a plan for *how* we will do it if we need to. Failing to do so would be negligent.

I don’t think this needs to start as a global coordination mechanism. The U.S. and China seem like all you’d need to begin with, and you can even throw in all the other countries that would obviously be swayed by the U.S. and China making such a commitment.

@viemccoy (OpenAI): I'm really proud to have signed this. This is the first of this sort of letter that I find meaningfully reasonable: calling on the government to do something that is not possible for the labs to do on their own. Creating the institutional infrastructure for a coordinated pacing of frontier progress gives us all the chance to prepare for what is coming next.

Recent events have shown that we need to dedicate more resources than we thought to alignment. I personally think alignment isn't an issue you solve once, but rather something we need to continually invest in through healthy environments and wise targets.

My hope is that pacing frontier progress will allow us to realize the importance of a healthy ecology of minds, and give us the chance to avoid mode-collapsing onto either a dangerous or grayslop future.

I'm not in favor of stopping, and I think there are a thousand beautiful futures waiting for us, co-existing in a Multipolar galaxy made possible by a pantheon of artificial superintelligences. But, I think this is a unique time where coordination could make an enormous difference when it comes to negating possible tail-risks, and this letter is the first thing I've seen that has given me hope that it is possible.

I know this won't sit right with everyone, particularly my accelerationist followers. To this, know that I stand with you on the side of eliminating death and disease. I will fight to ensure we don't pause forever, and I will not allow our world to fall into a bureaucratic hell of endless unachievable gates to proceed. For now, however, I think cool heads must prevail and I hope this letter will allow this to happen.

Jason Wolfe (OpenAI): OpenAI has said that humans should remain in control of AI development, and that decisions about the pace of progress should be made through democratic processes rather than left to individual labs. I strongly agree.

Progress is already moving very quickly. By default, competitive pressure rewards whichever company or country is willing to move fastest and accept the most risk. Recursive self-improvement could dramatically accelerate those dynamics, potentially beyond our

collective ability to understand progress, assess risks, and maintain meaningful human oversight.

We should start with stronger domestic transparency about frontier training, internal deployment, the pace of progress, and the risks and safeguards associated with increasingly capable systems. Democratic decisions are impossible if the public and policymakers don’t have the information they need to make them.

But transparency alone won’t solve an international race. We also need to build the technical and governance capacity for credible, verifiable international coordination. Maybe we never need to use it. But with many experts considering an intelligence explosion plausible within the next two years, I think it's urgent that we start building this capacity now.

Leo Gao (OpenAI): The world is locked in a deadly race towards an intelligence explosion, where AI’s ability to create better AIs reaches a critical point, like a runaway nuclear chain reaction. Since no individual actor is willing to stop unilaterally, to survive we must coordinate to slow down.

Geoff Penington (OpenAI): I signed. Coordinating a slowdown will be difficult and comes with a host of eg antitrust issues. But I used to wish it was easier to find big training wins and right now I kind of wish it was harder

Micah Carroll (OpenAI): At the current pace, every couple of weeks there will be new models which significantly increase the consequences of model misuse and misalignment. I worry that efforts to mitigate these risks may fail to keep up with the pace of development, and that margins for error will become increasingly small under international competitive pressures.

In the near future, we may urgently want to enact an internationally coordinated slowdown, or an indefinite ban on AI development. Attempting to build the trust and infrastructure for taking such actions on short notice seems simply prudent – why would we not at least try to have this option? I fear that in an international race to the bottom of AI development, it is likely that no nation will win, and we will all lose together.

Xerxes (Google DeepMind): I dedicated my career to AGI safety and alignment because I believe highly capable AI can go very well or very badly for humanity. We can test whether an AI behaves well. We cannot yet test whether it will keep behaving well when it matters. Until that changes, we need the option to coordinate a slowdown in AI capability growth.

Neel Nanda (Google DeepMind): I signed this. AI is progressing very fast, with incentives to go as fast as you can, even if there are risks. Coordinating a change of pace may be needed, but will be hard and needs prep, so ensuring there's the *option* is obviously good

I'm glad this is consensus across labs

Words From Others

Peter Wildeford (on MTS): I don't think anyone wants to stop AI now, but it's pretty clear the people building these AI systems are worried about the speed at which AI is accelerating, the speed at which these products are coming out, especially after last week's incident where an OpenAI model managed to escape OpenAI and attack another company. It seems like we don't have full control over our AI systems."

As these companies are potentially building towards superintelligence, towards recursive self-improvement, it seems very possible these systems might rapidly get away from our ability to control them. The way I'm reading this letter, it almost honestly seems like a call for help.

You have people from OpenAI, Meta, Google, Anthropic, co-founders, top scientists, all saying that this intense competitive pressure is pushing them to put out products faster than they know how to control, and we may not like where that ends up. It's been good so far, but we definitely need to tread very carefully.

Daniel Kokotajlo of AI 2027 and Plan A updates his distribution of possible futures, and is substantially more optimistic that we will choose wisely. As with my adjustments, presume this is a mix of the letter having impact and learning that the letter was possible and that someone went ahead and created it.

Daniel Kokotajlo: Updating my probabilities accordingly :)

Daniel Kokotajlo: Explainers & disclaimers:

--These probabilities are just my guesses. I made them as part of the research for AI 2040: Plan A,

https://ai-2040.com and you can find them in the supplement "Comparing possible plans." Other authors have somewhat different guesses which you can also find there.

--The black ones are the ones I made back then, the red and green are the updates I'm making now largely in response to today's news but also including other evidence that has accumulated over the last two months or so

--The numbers don't add to 100 because there are also some possibilities that don't fit into these five buckets

--Again, these are just guesses, I'm not confident etc. etc. I think it's valuable to put numbers on things and try to make them consistent with each other and adjust the numbers over time in response to evidence.

How likely you think we are to land on D vs. C depends a lot on where you draw the boundary of ‘at least a little bit.’ Like most trade-offs, I do not expect us to be maximally reckless, but I also do not expect us to often be willing to fully ‘burn the lead.’

Nick: I used to think slowing down was pointless, like slightly delaying a roller coaster. pausing at gpt2 for a year wouldn’t have changed anything. but I’ve been shocked by how much agents has sped up the best interp/alignment researchers I know. +6mo might decide how this goes

This is indeed a ‘middle path’ proposal, to give ourselves the option to do anything meaningful at all, and be able to collectively steer the future at all:

Daniel Faggella: there is an obvious terrible extreme of international tyranny and lock-in. there is ANOTHER obvious terrible extreme of "hurl uncontrollable AGI into the world in all directions"

the pacing statement is a movement to land near the middle / right now we need obviously that.

As always, there are many who cry ‘international tyranny’ at the thought of any form of even laying the groundwork for a deal or cooperation or collective action whatsoever, no matter its contents.

Or that will write off any such efforts as motivated by profits or delusion, or tries to equate this as ‘join forces with the anti-datacenter crowd.’

Consider your opinions noted, good sirs. We get letters.

I also threw out a reaction thread, for those who want a vibe check.

A Good Start

I believe the letter is on the Pareto frontier of ‘how much you say and how explicitly you say it’ versus ‘how many people and important people and labs sign the letter.’

It remains true that the letter is soft-pedaling compared to the actual situation as I see it, and there are many who would go farther than I would.

For thos