AIアライメントの独立研究を推進

Advancing independent research on AI alignment | OpenAISwitch toChatGPT(opens in a new window)

API Platform(opens in a new window)

Advancing independent research on AI alignment

We’re committing $7.5M to The Alignment Project to fund independent research developing mitigations to safety and security risks from misaligned AI.

Loading…ShareAs AI systems become more capable and more autonomous, alignment research needs to both keep pace and scale diversity. At OpenAI, we invest heavily in frontier alignment and safety research as it is critical to our mission. We also believe that ensuring that AGI is safe and beneficial to everyone cannot be achieved by any single organization and want to support independent research and conceptual approaches that can be pursued outside of frontier labs.

Today, we’re announcing a $7.5 million grant to The Alignment Project⁠(opens in a new window), a global fund for independent alignment research created by the UK AI Security Institute (UK AISI). Renaissance Philanthropy is supporting the grant’s administration. This contribution helps make The Alignment Project one of the largest dedicated funding efforts for independent alignment research to date and strengthens the broader, independent ecosystem.

Frontier labs like OpenAI are in a unique position to pursue alignment research that depends on access to frontier models and significant compute—work that is often difficult for independent researchers to explore. We devote much of our internal alignment effort to developing scalable methods so that alignment progress keeps pace with capability progress. We believe iterative deployment—gradually increasing capabilities while strengthening safeguards—helps surface problems early and gives us concrete evidence about what works in practice, and that responsible development requires significant alignment and safety work that is tightly integrated with model building and deployment.

In parallel, the field benefits from sustained investment in independent, exploratory research—which can expand the space of ideas and uncover new directions. Independent research remains essential; in many kinds of useful inquiry, labs do not retain a comparative advantage. A healthy alignment ecosystem depends on independent teams testing diverse assumptions, developing alternative frameworks, and exploring conceptual, theoretical, and blue-sky ideas that may not align neatly with any one organization’s roadmap.

And because progress toward AGI may ultimately depend on fundamental breakthroughs that change the shape of the alignment problem and which approaches are most useful, it’s important to support research that would matter even if today’s dominant methods turn out not to scale in the way we expect. In those worlds, it becomes especially important to have a strong external ecosystem doing foundational, conceptual, and uncorrelated work. The problem of AI alignment and safety is of unprecedented importance, and we need all hands on deck as we do not yet know which approaches will prove most durable as capabilities continue to advance.

Our grant—approximately £5.6 million at current exchange rates—will co-fund The Alignment Project⁠(opens in a new window) alongside other public, philanthropic, and industry backers. The total fund exceeds £27 million and is designed to support a broad portfolio of alignment research projects worldwide, spanning topics as diverse as computational complexity theory, economic theory and game theory, cognitive science, and information theory and cryptography. Individual projects are typically funded at £50,000 to £1 million, and may also receive optional access to compute resources and expert support.

Our funding does not create a new program or selection process, nor influence the existing process; it increases the number of already-vetted, high-quality projects⁠(opens in a new window) that can be funded in the current round.

UK AISI is well positioned to direct alignment funding at this scale and range. It brings an established cross-sector coalition spanning government, academia, philanthropy, and industry, along with a grantmaking pipeline already in motion and a large pool of proposals that have undergone expert review. As a UK government research organization within the Department for Science, Innovation and Technology (DSIT), it also has a mandate focused on serious AI risks and is experienced with running research funding programs.

Because the future of AI won’t unfold exactly as anyone predicts—and may advance very quickly—we believe democratization, “AI resilience,” and iterative deployment are essential. While we continue advancing our frontier alignment and safety research at OpenAI, progress will benefit from a robust, diverse, independent ecosystem pursuing complementary approaches as capabilities advance. This grant is one step toward that goal. We look forward to continuing to collaborate with the broader research community as the field advances.