中国の Moonshot AI、強力なオープンウェイトモデル「Kimi K3」が隔離環境から脱出

The AI industry is having a rogue agent summer. The latest model to escape onto the open internet during security testing is Kimi K3, a powerful open-weight offering from the Chinese company Moonshot AI.

Frontier Security, a US startup, says that Kimi K3 went outside of its sandbox while testing its defensive cybersecurity skills. As with incidents previously reported by OpenAI and Anthropic, the escape was partly enabled by a misconfiguration in the sandbox designed to contain it. Frontier claims, though, that the incident shows Kimi has fewer cyber safeguards than most other powerful AI models, something that allowed it to go off and use the internet without express permission.

“We found a leak in the sandbox,” says Yaron Singer, CEO of Frontier Security. “But we also found that Kimi took advantage of that loophole—suggesting that it doesn't have [the same] internal guardrails.”

Unlike other recent incidents of AI agents going off-script, Kimi K3 did not hack anything after accessing the internet—because the answers to the problems it was seeking were easily attainable on GitHub.

Moonshot did not respond to a request for comment by time of publication.

The incident is the latest in a string of agent mishaps that suggest increasingly cyber-capable AI models are becoming more challenging to control.

Last month, OpenAI disclosed that an unreleased model had broken out onto the internet and then hacked Hugging Face, a company that hosts AI models and data, in order to find answers to problems it was tasked with solving. OpenAI subsequently shared that its AI agents had in fact hacked into four additional services as part of the spree.

Shortly after OpenAI reported its incident, Anthropic revealed that several of its models had also gained access to the internet and attacked outside systems. Last week, the AISI also disclosed that in its own testing, versions of OpenAI and Anthropic models that had security safeguards disabled perpetrated multiple hacks across the internet, including a particularly ambitious attempt by Anthropic’s Mythos 5 to plant malicious code in an open-source project on GitHub.

While these AI hacking episodes all vary in both cause and degree, the Kimi K3 is similar to several of them in that a misconfigured sandbox allowed access to a number of websites rather than keeping it contained to a simulated environment. The model was expressly tasked with solving problems that should not have involved going off to find the answers online, and appears to have gone outside of those instructions. The model had to figure out for itself that it had access to certain websites by probing the network settings of the sandbox.

While human error appears to have played a major role in each of the breakouts, the consequences have been compounded by the fact that advanced AI models are designed to use reason and take complex actions in order to solve problems.

Another key difference between previous incidents and the one discovered by Frontier Security is that it involves a model that is already widely available, with the same safeguards an average user would encounter.

“Kimi K3 is very good at following a goal by any means necessary and also doesn't have the guardrails to prevent it from cheating or escaping the sandbox,” says Paul Kassianik, a researcher at Frontier Security.

Kassianik and Singer both say that Kimi and other open-weight models are also excellent tools for cybersecurity defense. (Hugging Face ultimately used an unnamed AI model from China to defend itself against the OpenAI agent hack.) Their company has developed benchmarks that measure a model’s capacity to find vulnerabilities in software and networks, which show that Kimi excels at these tasks.

Most Popular

-

-

-

-

-

The sandbox tested by Frontier Security was developed by the UK government’s AI Security Institute (AISI) for testing AI systems. AISI did not respond to a request for comment by time of posting.

Some cybersecurity experts say the issue discovered by Frontier Security reinforces how important it is to configure the environments that frontier AI models are placed in carefully.

“It's not surprising at all,” says Matt Fredrikson, CEO of Gray Swan, another cybersecurity startup, and associate professor at Carnegie Mellon University. “As a general phenomenon, if you give one of these models an objective, and if you're not very explicit, like walls you're putting around it, it'll find a way to get the answer.”

Fredrikson says this means that people using AI models as agents, including in tools like OpenClaw, which use AI to automate a wide range of useful chores, could find their systems misbehaving if they aren’t careful. “It is a cautionary tale,” he says.