智譜 AI、GLM-5.3 のセキュリティ結果を詳細解説

Zhipu’s release note for GLM-5.3 contains a sentence that did not make it into most of the coverage. Describing its own cybersecurity results, the Beijing company writes that capability “is growing fastest exactly where we are furthest behind.”

Zhipu, which also trades as Z.ai, is one of a handful of Chinese labs releasing models that compete with the American frontier. On August 14, it launched GLM-5.3, a coding-focused model, and published a technical release note setting out how the model performs against its rivals. That note is the source for everything reported here.

The claim that travelled was about security. Alongside the coding results, Zhipu said GLM-5.3 had become unexpectedly good at finding software vulnerabilities, scoring 84.5% on a benchmark called CyberGym against 83.8% for Anthropic’s Mythos 5 and 83.6% for OpenAI’s GPT-5.6 Sol. Headlines followed reporting that a Chinese model now out-finds the American ones at bug hunting.

The reason that lands harder than a usual benchmark result is what vulnerability discovery has become. A model that can read a codebase and locate exploitable flaws is useful to a defender auditing their own software and useful to anyone doing the same to somebody else’s. Anthropic’s equivalent work sits behind restricted access for that reason, while Zhipu intends to publish GLM-5.3’s weights for anyone to download.

Zhipu’s own release is more measured than the coverage it produced. The CyberGym number is real, and it is in the paper. It is also the narrowest of the three cybersecurity results the company published, and Zhipu is upfront that the other two go the other way.

Three benchmarks, three different pictures

CyberGym starts from source code the model can read and tests whether it can find a vulnerability and confirm the flaw is genuine. That is the result that travelled, and the margin is seven tenths of a percentage point.

ExploitBench asks something harder, requiring the model to reason about a real vulnerability and how it would be exploited. GLM-5.3 scores 54.4%, more than double its predecessor’s 24.4%. Mythos 5 scores 78.0% and GPT-5.6 Sol 76.5%.

ExploitGym counts how many exploitation tasks a model finishes inside a fixed time budget. GLM-5.3 completes 105 tasks in two hours and 130 in six. Mythos 5 completes 181 and 247.

Those two results have been reported thinly, and they are the ones that describe the gap. Finding a flaw and building a working exploit from it are different jobs. Zhipu’s reading is that the further along that chain a test sits, the further behind its model is, and the company says so in the release rather than leaving it to be discovered.

imageZhipu’s own comparison across the three cybersecurity benchmarks. GLM-5.3 leads on CyberGym, the vulnerability discovery test, and falls behind Anthropic’s Mythos 5 on both exploitation measures. Source: Z.ai.

Which Anthropic model, and why it keeps changing

Part of the confusion in the coverage comes from Zhipu comparing three different Anthropic models in three different places. The main benchmark table sets GLM-5.3 against Opus 4.8. The performance charts use Fable 5. The cybersecurity section uses Mythos 5. Anyone reading quickly comes away with a single comparison that does not exist.

On coding, the picture is mixed rather than dominant. GLM-5.3 leads Opus 4.8 on some tests and trails it on others, and Zhipu states plainly that its model remains behind Claude Fable 5 on the company’s own internal coding benchmark.

How the tests were run

The methodology footnotes contain something the summaries skipped. Zhipu evaluated GLM-5.3 on CyberGym, ExploitGym, ExploitBench, Terminal Bench and several other tasks inside Claude Code 2.1.207, Anthropic’s coding agent.

That is not improper. Using a common harness across models is how a comparison stays fair, and Zhipu documents the settings it used. It is worth noticing anyway. A Chinese open-weights model’s frontier claims are being measured through American agent software, which says something about where the tooling layer sits in this competition that the model scores do not.

Two further details deserve attention before the CyberGym result is treated as settled. The score is a single run, reported as pass@1 across 1,507 tasks, with no variance figures given. A gap of seven tenths of a point between two single runs is not a gap anyone should lean on. And the ExploitGym time budgets were normalised using throughput rates from Artificial Analysis, with rescaling factors listed for GLM-5.3, Kimi K3 and Qwen3.8 Max, but not for Mythos 5.

The vulnerability count and the number that is missing

Beyond the benchmarks, Zhipu says it worked with security teams in China to run its models against real codebases, identifying 2,436 vulnerabilities across 269 open-source projects. The severity split is 107 critical, 990 high, 1,286 medium and 53 low. The oldest flaw dates to 1981, and the average vulnerability had been sitting in code for 26.6 years before it was found.

imageZhipu’s disclosure summary. The panel labels 1,097 findings as critical and high, matching the severity breakdown of 107 critical and 990 high. The body text of the same release describes the figure as medium-to-high. Source: Z.ai.

One discrepancy is worth carrying carefully. Zhipu’s summary panel labels 1,097 findings as critical and high, which matches the severity table. The body text of the same release describes those 1,097 as medium-to-high. Several outlets have reproduced the second version.

The count also arrives after what Zhipu describes as expert review, screening and deduplication, so the raw model output is not what is being reported. Of the 2,436 findings, 53 have been publicly disclosed, and 2,383 remain under embargo. The release does not say how many were previously unknown, and it does not say how many were independently reproduced. Those are the two figures that would turn a volume claim into a capability claim.

What matters more than the benchmark table

Two things in the release have longer consequences than the CyberGym margin.

The first is efficiency. Zhipu reports GLM-5.3 reaching 31.4% on its internal coding benchmark at around 50,000 output tokens per task, against Opus 4.8 at 29.5% using 120,000. Slightly better work for less than half the tokens is a cost argument, and cost determines whether security teams outside the largest budgets can run these tools at all.

The second is distribution. Zhipu says the weights will be published once safety evaluation and hardening are finished. That has not happened yet, and until it does, the open-weights claim is a commitment rather than a fact. If it holds, a model with documented vulnerability-discovery capability becomes something any team can download and run locally, including in markets that will never have access to an export-controlled American model.

The weights are due at the end of August.

See also: Anthropic walks into the White House and Mythos is the reason Washington let it in

Want to learn more about AI and big data from industry leaders? Check out AI & Big Data Expo taking place in Amsterdam, California, and London. The comprehensive event is part of TechEx and is co-located with other leading technology events including the Cyber Security & Cloud Expo. Click here for more information.

AI News is powered by TechForge Media. Explore other upcoming enterprise technology events and webinars here.

The post Reading Zhipu’s GLM-5.3 results past the headline number appeared first on AI News.