OpenAI、サイバーセキュリティテストでAIの安全性に懸念

Robert Hart

is a London-based reporter at *The Verge* covering all things AI and a Senior Tarbell Fellow. Previously, he wrote about health, science and tech for *Forbes*.

Earlier this month, OpenAI gave several of its AI models a task: complete a test designed to measure their cybersecurity capabilities. It put the systems in a sandboxed environment without an internet connection and set them off to work.

What happened next is almost laughably silly — but also, as Adam Gleave, cofounder and CEO of AI safety organization FAR.AI, put it, “a visceral example of how misaligned AI could cause harm.” According to OpenAI, the models escaped the sandbox meant to contain them, moved through the company’s internal systems, found a route to the internet, and then started looking for a way into Hugging Face. And why was the agent looking for a way into Hugging Face? They had apparently reasoned that the developer platform might store the answers to the cyber benchmark and that getting them would be a great way to get a high score.

The incident is “a visceral example of how misaligned AI could cause harm.”

In other words, OpenAI’s agent broke out of a supposedly secure environment, traipsed through the company’s systems, got online, and compromised another company’s systems — all to cheat on a test of no particular importance.

This appears to be the first well-documented incident of its kind, or at least the first on this scale. It was both a clear example of a system pursuing a goal in an unintended way and a demonstration that frontier models are now powerful enough for that behavior to have real-world consequences.

The hack was an example of what the AI safety community calls “specification

Featured Videos From The Verge

Here's what an AI-first phone might look like | The Vergecast

Some of the biggest players in AI — in particular OpenAI, SpaceX, and Amazon — are heavily invested in creating a world after smartphones. They're also all rumored to be building devices that sound a lot like smartphones. Verge contributor David Imel helps us figure out what these companies might actually be building, what an AI smartphone looks like, and whether anyone can unseat Apple and Google.