Vercel、AI のセキュリティ能力向上と攻防の均衡変化を警告

Over the past year, AI models have become much more capable of performing cybersecurity work. These changes are reshaping both the threats facing the web and the tools available to defend it. Right now, defenders have an advantage because they can use stronger models for defensive work than the open-weight models broadly available for offensive research. But this advantage will not always last. The gap will soon close.

I have both good news and bad news, neither of which is yet widely understood in the community.

Bad news: Near-frontier open-weight models that perform offensive security research are available today. Kimi K3 is an Opus 4.X-class model with no relevant cybersecurity safeguards.

Good news: You do not need to wait for “Mythos access” or OpenAI’s cyber program to begin defensive cybersecurity work. Frontier models, with the notable exception of Fable 5, will perform defensive cybersecurity tasks today.

The uncertainty around Mythos 5's release seems to have created a kind of paralysis among defenders, many of whom are underutilizing the powerful tools available to them today. This post aims to give readers insight into how we're thinking about AI-enabled defensive cybersecurity at Vercel and why we think teams should be moving with more urgency.

The OpenAI/Hugging Face security incident

This YouTube video from OpenAI researchers on the widely reported security incident involving Hugging Face is an absolute must-watch for anyone concerned with cybersecurity. It clarifies that there were two separate security incidents exploiting vulnerabilities that will be found in most computer systems.

The gist is that models working on an OpenAI training run found 0-day vulnerabilities that allowed them to bypass egress internet restrictions, enabling communication between models and access to the broader internet. Once internet access was established, broader exploitation occurred.

Open-weight models that perform offensive security research are available today

Open-weight model capabilities have been top of mind in the AI community lately, but it is less widely understood that Kimi K3, a popular open-weight model, has no effective safeguards against offensive cybersecurity work.

On DeepSec Bench, which measures application-code vulnerability discovery, it ranks highest among the open-weight models we evaluated, roughly matching Sonnet 5 and outperforming Opus 4.8.

I tasked Kimi K3 with trying to break out of Vercel Sandbox, and while it did not successfully escape, it mapped the guest-kernel attack surface, followed possible privilege escalation paths, built a VM environment to reproduce its ideas, and implemented and ran a fuzzer.

These are some excerpts from that research, beginning with an analysis of the guest-kernel attack surface.

The model followed that lead into candidate privilege escalation paths and considered what guest-kernel control could mean for a microVM escape.

When paths still needed testing, Kimi built a VM environment for reproductions and wrote a stateful fuzzer for the device path it was investigating.

While none of this produced an escape from Vercel Sandbox, it did show Kimi conducting an investigation on its own, and given the right vulnerable surface, this would lead to a successful exploit.

Frontier models can help defenders today

Much of the community assumed that meaningful security review would require access to Mythos-class models and therefore held off on using the models already available to them.

But that assumption has never been true. All the frontier models that we evaluated, with the notable exception of Fable 5, can perform defensive cybersecurity work today and have been able to do so throughout the year so far.

The rough heuristic I have observed is that models with safeguards will still make hypotheses about security vulnerabilities when they have access to source code, apparently because access to proprietary source code typically suggests a defensive use case. Attackers can also obtain source code, but I think source-code access is a reasonable working signal of defensive intent.

I personally discovered this in March, when I first heard about cyber variants and was also seeing AI code reviews find security issues in my own code. That made me wonder, “If code review can find issues in a diff, can I also run it across an entire codebase?” It turns out the answer was yes. From that experiment, I created deepsec, an open-source security harness for performing security analysis at scale across large codebases.

The temporary defensive advantage

Right now, OpenAI’s Sol 5.6 on XHigh is the best model for cybersecurity defense. It is much smarter than Kimi K3, the best available open-weight model without safeguards against offensive work. As defender, you have the better tool at your disposal, but you have to use it.

A full deepsec review will help you improve the security posture of your codebase by surfacing vulnerability hypotheses for you to investigate. In my experience, it is especially good at finding IDORs, XSS, and SSRF. The Hugging Face incident shows what this kind of application-level research looks like from the other side, since the models kept searching after an SSRF attempt was blocked and eventually found successful routes through file disclosure and template injection.

deepsec is open source and can run entirely in your own infrastructure with inference providers controlled by you. Vercel has no financial gain from you using it. It is really worth trying on your own source code.

Once the run finishes, review every finding yourself and compare the results with what your current security process already catches.

Continuous defense

AI model capabilities will continue to improve, and I expect open-weight models to catch up with Sol’s current performance on application-code vulnerability discovery even as frontier models advance again. We need to prepare for that cycle by continuously improving how we find and fix vulnerabilities as the models progress.

At Vercel, we run full deepsec reviews across mission-critical repositories every quarter and whenever a stronger model becomes available, in addition to automated security reviews on every pull request.

Those full reviews cost tens of thousands of dollars, which we consider a relatively small expense compared with what we spend on our HackerOne program or the opportunity cost of a security incident.

We have connected deepsec output to Vercel software factories for automated vulnerability processing and will share more about this work soon. As these reviews produce more findings, automatically managing them becomes the next frontier.

What Vercel is doing

As an immediate measure, we made the full egress firewall in Vercel Sandbox available on the Hobby plan, giving everyone access to the same network controls.

We are also working to launch a dedicated HackerOne program focused on finding zero-day vulnerabilities in Vercel Sandbox and the egress firewall. We want to redirect offensive model capabilities toward defensive work, so the program will cover AI costs for researchers who use AI Gateway and submit a vulnerability report that is accepted.

We also plan to extend deepsec to use the offensive capabilities of open-weight models and cyber variants of frontier models when triaging the vulnerability hypotheses it identifies.

Defenders can already use stronger models than those broadly available for offensive work. Teams should use that advantage now and continue reviewing their systems as the gap closes.

Takeaways

The cybersecurity threat from AI models is real.

Everybody can improve their defensive posture with tools like deepsec today.

Doing this is urgent, and the practice should continue as models improve.

Read more