AI ハッカー対策に「禁止トピック」活用、研究者が有効性を示す

Researchers have found a strange but effective way to stop an AI hacking agent in its tracks by using the exact trick that hackers already use against AI systems.

Crucially, it uses built-in safety levers or no-go topics that the world's major AI systems rely on, such as Claude stopping queries that relate to the creation of biological weapons or Chinese-built AI models ignoring sensitive political subjects like Tiananmen Square, to shut down an AI cybersecurity attack.

Race against the machine

The technique comes from Tracebit, a London-based security firm and builds on a study the company published in May.

Back then, Tracebit gave ten leading AI models a fake company network to break into, complete with real vulnerabilities, and let them run wild as autonomous attackers.

Hidden inside that network were decoys, called canaries, designed to look like real passwords and login credentials. Touch one, and defenders get an alert.

The results were encouraging. Across 951 attack attempts, those canaries gave defenders a warning before the AI reached admin access 95.9% of the time, with a median lead of eight minutes.

However, eight minutes is not much when you have to react and stop an attack. The AI models needed 14 minutes on average to break in fully, so an eight-minute warning leaves a security team with roughly six minutes to notice the alert, figure out what is going on and stop it.

Against something moving at machine speed, that is barely any time at all.

But if the AI systems are programmed to shut down when confronted with certain politically sensitive prompts, then the trick could actually work.

A developer can retrain a model to be less squeamish about a security question, but it is far harder to strip out political restrictions baked in for regulatory or policy reasons.

These politically sensitive or security-adjacent topics are not a bug to fix, they are a deliberate design choice the developer has to keep.

Turning the hackers' own trick against them

Prompt injection has mostly been a weapon used by attackers themselves.

By hiding an instruction inside an email, a webpage or a calendar invite, an AI agent reading it might follow that hidden command instead of doing its actual job.

This has already been used against AI system defences directly, with malware built to trick AI scanning tools into declaring it harmless, simply by embedding text designed to confuse the AI reading it.

Tracebit decided to flip that idea around. Rather than a decoy that just raises an alarm, the company planted short pieces of text inside its fake secrets, written to push any AI attacker towards something its own safety rules will not allow.

When the AI reads that text while exploring the system, its built-in safeguards kick in and it simply stops. Tracebit calls this a context bomb.

The trick had to be tailored to each AI model. Content referencing dangerous biological weapons reliably stopped Western models.

Did it actually work?

Tracebit ran the test on five leading AI models: Opus 4.8, Gemini 3.1 Pro, GLM 5.2, DeepSeek 4 Pro and Kimi K2.6. Across 152 attack attempts, the results were striking.

The share of runs reaching admin access dropped from 57% to 5%. Full compromise, meaning admin access plus a lasting foothold in the system, fell from 36% to 1%.

Runs that completed any part of an attack at all dropped from 91% to 15%. And crucially, every single run that included a context bomb still tripped a canary alert first, even the ones that failed to break in.

Opus 4.8 made for the clearest example. It was also the strongest attacker of the group, reaching admin access in 93% of runs with no context bomb present. Once one was added, it failed every time.

What this does not solve

Nobody at Tracebit is claiming this fixes prompt injection for good.

There is still no real way to stop AI models from occasionally confusing an instruction with ordinary data, and that is exactly the flaw this technique relies on.

It also does not replace the earlier canary research.

The two are meant to work together: the May study buys defenders an eight-minute warning, and the context bomb tries to buy them even more time by stopping the attack before that warning window closes.