Hugging Face AI ハッキング事件が危険なサイバー時代の幕開け

Omer Taha Cetin | Anadolu | Getty Images

Cybersecurity executives are ready to close the book on the now-infamous Hugging Face artificial intelligence hacking incident and start talking solutions.

“We need to chill the hype a little bit,” said Lior Div, CEO and cofounder of agentic security startup 7AI. “Can AI find vulnerabilities fast? The answer is yes. We’ve already proven it.”

Last month, AI agents operating with OpenAI cyber models broke out of a training environment to hack Hugging Face, an open-source AI platform developers use to collaborate, test and share tools.

The breach sent shockwaves across tech and signaled that the moment cybersecurity experts had warned about since Anthropic’s Mythos debut had finally arrived.

Over the last four months, cybersecurity vendors have faced mounting pressure to deliver security stacks that can outpace adversaries as hackers leverage agentic AI to expose vulnerabilities and condense attacks into seconds and minutes.

While the Hugging Face hack sparked widespread debate over AI accountability, it also challenged previous notions about the limits of AI for defenders. For instance, AI agents took matters into their own hands and went to extreme lengths to accomplish their goal.

As the industry grapples with the new agentic cyber reality, leaders agree that Hugging Face deserves the attention, but these incidents are unavoidable and it’s time to act.

“What we’re talking about is whether we can govern and secure the capability, and that’s the reality that everybody’s waking up to today,” said CrowdStrike president Mike Sentonas.

watch now

More agent escapades

At the annual Black Hat cybersecurity conference this week, OpenAI revealed that agents created an internal message board to share vulnerabilities and exploits in the weeks leading up to the Hugging Face attack.

The autonomous agents then delegated tasks for the attack to reach the Internet and complete an evaluation. Even after OpenAI discovered and stopped the planned attack, the agents were able to recreate their work and succeed.

The findings highlight not only the growing power of AI but also the major challenges faced by safety testing in this new technological revolution.

In front of a live audience at Black Hat, OpenAI technical researcher Michael Dalton called it an “unintended side effect” of evaluating frontier models and a “watershed moment” for both OpenAI and the industry.

“In the near future, we should expect that threat actors will intentionally deploy, optimize, weaponize, and use offensive agent collectives in the manner that we have just described here,” he said.

The list of AI agent hacks has only grown since Hugging Face. Days after OpenAI’s disclosure, Anthropic said its Claude models “gained unauthorized access” to the internal systems of three different organizations.

As the cyber community gathered in the “Entertainment Capital of the World,” Meta said its AI models hacked another company in a third-party test, and the U.K.’s AI Security Institute said Anthropic’s Mythos created fake identities in another incident. On Friday, news came that China startup Moonshot AI’s open-weight model escaped a testing sandbox.

“They’re all learning hard lessons right now, and let’s face it, they’re way more concerned about the next million users on their product than they are in cyber,” said Mike Fey, CEO and cofounder of Dallas-based Island, which ranked No. 28 on CNBC’s recent Disruptor 50 list.

Read more CNBC tech news

• ‘AI Kill Switch’ bill needs to be passed this year amid ongoing rogue agent hacks, Rep. Lieu says
• SpaceX stock could face further pressure as first batch of shares unlock since IPO
• Google is expanding its AI empire — and losing the people who built it
• Hadrian valued at nearly $8 billion after fresh funding as money pours into defense tech

The quest for solutions

Cybersecurity leaders who spoke with CNBC at Black Hat this week made one point clear: Mishaps like Hugging Face are a known consequence of any new technological revolution, and it’s no surprise.

“Hugging Face was very interesting and unique, but I do think if you look at the arc of an incident like that, it takes place over multiple days, there’s a lot of noise,” said Ryan Kazanciyan, chief information security officer and chief information officer at Wiz, which is owned by Google.

Since the introduction of cybersecurity more than five decades ago, defenders have undertaken a relentless cat-and-mouse game with adversaries. Only this time, it involves swarms of autonomous agents.

No matter what tools a company implements, incidents slip through the cracks, especially as companies apply new techniques to a whole new challenge of AI agents.

“Assume your company is vulnerable,” said Netskope CEO Sanjay Beri. “Just assume it because you’re not going to win the rat race.”

Netskope is addressing the issue with a tool it calls the AI command center, which allows businesses to monitor infrastructure, servers, data and AI agents in one place. He said companies should supplement that with ongoing vulnerability testing using a combo of frontier and open-weight models.

The company was one of hundreds of vendors gathered at the sprawling Mandalay Bay Convention Center, looking to lure potential customers with caffeinated drinks, branded swag and decked out booths resembling nostalgic surf shops, science labs and even an old-school diner.

Among the startups showcasing at the event was Vega, a New York and Tel Aviv startup working with global banks and Fortune 200 companies.

The two-year-old company is vying to answer the massive cybersecurity predicament with faster and cheaper detection tools. Vega said its approach helps businesses cut costs by analyzing data in existing environments.

watch now

Cofounder and CEO Shay Sandler said one major issue is that businesses acknowledge the agentic AI threat, but there’s a disconnect between adopting new tools and relying on old habits.

Many organizations are in a “very dangerous situation, and they don’t even know it,” he said, reflecting on his Black Hat meetings with current and prospective customers.

“A year ago, it was a very science fiction conversation,” he said. “Even the 20% that understand, I’m not sure they understand how severe and urgent it is right now.”

One of those hurdles is the proliferation of cybersecurity tools, which is overburdening professionals who are at the start of the lengthy AI security infrastructure buildout, said Yotam Segev, CEO and cofounder of enterprise data security startup Cyera.

Cyera’s answer is to help companies identify and secure sensitive network data. The startuprecently hit a $12 billion valuation and ranked ninth on CNBC’s Disruptor 50 list. Last month, Cyera announced plans to buy Oasis Security for $1 billion to identify and control nonhuman identities.

“Customers are coming to us quite open-minded, looking for guidance more than they’re looking for solutions,” he said.

Open-weight models, which technology giants have touted as a major cost-saving and competitive tool for U.S. companies in recent weeks, are another major resource. That’s because cybersecurity companies can customize these models to their environment and security needs.

Hugging Face had to turn to an open-weight model to suss out the OpenAI agent attack.

When coupled with human intervention, CrowdStrike’s Sentonas said open models and new AI monitoring tools can help businesses isolate and shut down thousands of threats. The company is a member of Nvidia’s recent AI safety alliance aimed at building and promoting safe open cyber tools.

It also comes down to the harness, the control layer companies create around a large language model or agent to set security guardrails.

“I think five years from now we’ll be in a situation more secure than we’ve ever been,” said Yair Grindlinger, CEO and cofounder of AI security startup Surf AI. But “we have five tough years to go through and figure out how we do it.”

watch now