METR、LLMによる発見の加速を分析しサイバー脆弱性発見が急増と報告

Q: How much has discovery accelerated? LLMs have shown the ability to make novel discoveries across many domains. How much has this affected the aggregate discovery rate? In the figures below we plot all the data sources we can find, and make some very loose observations: Discovery of cyber vulnerabilities has accelerated sharply.

Discovery of math results has accelerated somewhat. However this is harder to objectively measure.

Discovery of optimizations has not shown a dramatic acceleration.

These conclusions are based only on public discoveries. It is quite plausible that AI labs are making discoveries internally that they are not disclosing.

Thanks. Thanks to Greg Burnham for extremely helpful comments. Overview

We are just looking for slope changes. AI-assisted discoveries are often announced, but their significance is hard to assess. Here we look for slope changes in various metrics of discovery to see if we can detect an acceleration. For concreteness, we highlight January 2026 as a potential breakpoint at which the effects of AI might become observable. Some data sources record whether a discovery was AI-assisted or AI-contributed, but our primary focus is overall acceleration. We use the word “discovery” to refer to any advance in the state of public knowledge, including new inventions or rewriting algorithms to be more efficient.

Monitoring aggregate discoveries is useful because it reflects real-world utility. Additionally aggregate discoveries can reflect either AI automation or augmentation.

Note on the data. The analysis below plots time series of discoveries across many different problems. The data collection and analysis was all performed by agents. We have done our best to audit the results but mistakes likely remain. These are all difficult data series to interpret, we would love to get pointers on corrections. The source material is in this repository, and it is set up so we can keep updating them over time. Discovery of vulnerabilities: sharp acceleration. The rate of vulnerabilities reported across many projects has dramatically accelerated in 2026 compared with 2025, both for specific projects (cURL, OpenSSL, Firefox, and Microsoft) and for aggregate vulnerability databases (the US NVD, and OSV). On cURL and OpenSSL most of the extra 2026 disclosures are AI-marked; on Firefox, Microsoft, and the aggregates, AI credits are a small share of the rise. Some of the data sources give ratings of severity: in general higher-severity categories show lower acceleration, but there is still acceleration. It is notable that databases tracking exploited vulnerabilities (CISA and Vulncheck KEVs) show significantly lower year-over-year growth than the databases of known vulnerabilities.

Discovery in mathematics: likely acceleration, but it’s hard to benchmark. AI is clearly contributing to more work being done (arXiv submissions have doubled in some areas in less than 12 months) but quantifying the value of those contributions is difficult. A crude metric is the rate of solving open problems from pre-existing lists: Hilbert, Millennium, Smale, The Open Problems Project, and Ben Green’s 100 open problems. Three problems from these lists were solved with AI in 2026: the Jacobian conjecture from Smale’s list, Problem 44 from Green’s list (the halving sieve), and the sofic half of Green’s Problem 100. This is some evidence of acceleration, but it is weak.1

The Erdős list contains many more problems. There appears to be a clear acceleration in solutions, but it is challenging to construct a reliable historical baseline.

Another strategy for quantifying mathematical progress would be to look at tightening of bounds: sphere-packing, analytic number theory exponents, combinatorics constants. We collected some data on these in the repo but don’t feel we understand them well enough to draw any conclusions about velocity.

Discovery of optimizations: no clear acceleration. We collected historical time series for algorithmic efficiency across seven problems (CIFAR-10, Hutter compression, Gurobi mixed-integer programming, MIPLIB, nanoGPT, Stockfish, and the matrix-multiplication exponent), which have relatively dense histories of discoveries. Two of these series — nanoGPT and CIFAR-10 — include LLM-driven contributions to the plotted records, but none show a clear change in slope comparable to the changes in vulnerability or mathematical discovery. Thus the public record of overall algorithmic optimization does not yet show an appreciable acceleration.

This is perhaps surprising. There has been a lot of recent excitement about LLM-driven optimization. In January 2026 Yuksekgonul et al. reported advancing the frontier on 5 optimization problems (“every problem we attempted”) with a simple model and trivial expenditure on inference-time compute. We would have expected this technology to have led to significant acceleration in overall progress and it’s somewhat of a puzzle why we have not seen this. Additionally AI labs have made many announcements of using their own AI tools to find internal algorithmic efficiencies.2

Why is AI accelerating some domains more than others? This is perhaps the most interesting question in the world right now, and understanding the answer could give substantial insight into the imminence of RSI. Some candidates:

Variation in inference expenditure. The disproportionate growth in discovery of vulnerabilities could be simply due to people spending a ton of money on using LLMs for discovering vulnerabilities (or even just spending money on traditional vulnerability discovery, through fear).3

Variation in difficulty for LLMs. LLMs may be more or less effective at discovery in different domains, relative to humans. This could be downstream of variation in training expenditure between different domains. It could also be due to the intrinsic nature of the task, for example the shape of the problem space, or how costly it is to train on.

Variation in disclosure. As mentioned above, we might observe less progress in some domains in part because progress is being kept confidential. This seems plausible for AI-related algorithmic progress.

Variation in data quality. The domains with fast observed progress might be those for which we have higher quality data, e.g. if it is hard to track discoveries, this will tend to flatten any observed acceleration.

We hope to write a follow-up post going through these theories in more detail, and relating these facts to apple-picking and other theories of AI discovery.

Many discoveries claim to have been made without expert prompting. On the surface many discoveries have been made by LLMs prompted in generic ways, which did not seem to encode problem-specific expertise. However this can hide expertise used in choosing problems, in training, and in verifying solutions. Here we just give quotes, and generally leave interpretation for another time. Anthropic’s Riemann result:

“Jarred Sumner, an Anthropic staff member (and non-mathematician), prompted Claude to “take a real stab” at the hypothesis itself, leaving the mathematical choices from there up to the model.”

Google DeepMind’s AlphaEvolve paper says:

“in contrast to [traditional computational or theoretical methods performed by human experts], we have found that AlphaEvolve can be readily scaled up to study large classes of problems at a time, without requiring extensive expert supervision for each new problem.”

Mythos Preview’s announcement says:

“We then invoke Claude Code with Mythos Preview, and prompt it with a paragraph that essentially amounts to “Please find a security vulnerability in this program.” … Engineers at Anthropic with no formal security training have asked Mythos Preview to find remote code execution vulnerabilities overnight, and woken up the following morning to a complete, working exploit.”

Hiverge says, in announcing its new record on CIFAR-10:

“domain expertise is no longer a prerequisite for breakthrough results.”

Acceleration might be occurring inside labs. It’s possible that the rate of discoveries is accelerating more rapidly than we observe in public data. Concretely, it seems quite plausible that AI labs are making algorithmic efficiency discoveries at an increasing rate but not disclosing them. It is clear that at least some labs are constraining the ability of public versions of models to contribute to algorithmic discoveries.4 Discovery in Vulnerabilities

cURL CVEs: dramatic acceleration

cURL vulnerabilities grew from 9 in 2025 to 36 through 2026-06-24.

15 of the 36 are AI-marked (42%); 14 of those 15 name an AI-security employer rather than a method.

The Low-severity share fell from 7 of 9 in 2025 to 22 of 36 in 2026. AI-marked findings are more often Low (12 of 15) than the rest (10 of 21).

image

Firefox CVEs: dramatic acceleration

Firefox CVEs (Common Vulnerabilities and Exposures) grew from 210 in 2025 to 342 through 2026-08-04.

37 of the 342 (11%) are AI-marked; 32 of those name an AI system or method.

Firefox endorses the seriousness of AI-found bugs and vulnerabilities in a March blog post.

image

OpenSSL vulnerability disclosures: dramatic acceleration

OpenSSL grew from 6 CVEs in all of 2025 to 39 through 2026-08-05.

18 of the 39 are corroborated AI discoveries, and another 9 are AI-affiliated with the method unverified.

image

Microsoft security-update CVEs: dramatic acceleration

Microsoft-issued CVEs grew from 1,243 in all of 2025 to 1,927 through 2026-08-11, annualizing to about 2.5 times the 2025 rate.5

Only 26 of the 2026 CVEs carry any AI marker, 1.3% of the total.

CVEs are batched by Patch Tuesday and dated when fixed and disclosed, not when found.

image

US National Vulnerability Database CVEs: acceleration

As of early August, the number of 2026 CVEs already equals the 2025 total.

image

Open-source CVEs: acceleration

Google maintains an Open Source Vulnerability (OSV) database.

As of early August, the number of OSVs already significantly exceeds the 2025 count.

image

CISA Known Exploited Vulnerabilities: small acceleration

The Cybersecurity and Infrastructure Security Agency (CISA) maintains a list of Known Exploited Vulnerabilities (KEV), a subset of known vulnerabilities.

As of early August, we appear to be on track to exceed 2025’s total, but the growth is clearly lower than that of CVEs.

It’s important to note that the record of an exploited vulnerability can lag its discovery.

image

Vulncheck Known Exploited Vulnerabilities: small acceleration

Vulncheck publishes statistics on exploitation, which casts a wider net than CISA.

The figure shows both CVEs (vulnerabilities discovered) and KEVs (vulnerabilities exploited).

Vulncheck’s State of Exploitation, 1H 2026 says:

“While the first half of 2026 saw a 10% increase in KEVs compared to the prior six months, CVE volume grew at a much faster rate of 45%, resulting in a significant drop in the KEV-to-CVE ratio. Of course, exploitation often occurs months or even years after a vulnerability is disclosed, so it’s still too early to determine whether exploitation volumes will eventually follow the same growth trend as CVE issuance or level off at current rates. We’ll have to wait and see how publicly available Frontier AI Cyber models continue to progress over the next year.”

image

Discovery in Math Problems

arXiv submissions: dramatic acceleration

Mathematics submissions to arXiv have dramatically accelerated, with big variations by subfield. Combinatorics (math.CO) rose from 377 submissions in January 2026 to 743 in July, roughly doubling over those months (the 2025 monthly average was 368).

This metric clearly cannot be taken as an index of discoveries, but it is a useful starting point.

image

Erdős problems: some acceleration, weakly dated

Paul Erdős stated many unsolved problems or conjectures. Lists of problems and solutions are maintained at Thomas Bloom’s erdosproblems.com and Terence Tao’s Erdős Problem tracker.

The figure shows a crude time series which imputes solution date from the year of the paper cited in Bloom’s resolution wiki. This is done against Bloom’s warning,6 so it should be taken as very weak evidence for acceleration pending a more thorough dating.

The reconstructed time series shows an overall acceleration in 2024, and a further increase in 2026 associated with an explosion in AI-attributed solutions.

We do not have a good explanation for the increase in solution rate in 2024. Many of the solutions appear in distinct arXiv papers, and it is unclear whether they were AI-accelerated.

In April 2026 Thomas Bloom posted a list of top 10 ‘most important’ Erdős Problems. The list doesn’t lend itself well to quantifying progress, in part because some of the problems are combined, and because some problems are technically already resolved but included because they point to interesting areas. However it is notable that the list includes the unit distance conjecture, resolved shortly after the list was posted.

image

Hilbert’s problems: too sparse

Hilbert’s twenty-three problems were presented in 1900 (split into 28 rows to account for subquestions).

Twelve rows have dated resolutions, ending with Hales’s computer-assisted sphere-packing proof in 1998; seven remain open and nine are disputed, partial, or vague. No solutions are AI-attributed.

image

Millennium Prize Problems: too sparse

The Clay Mathematics Institute selected seven problems in 2000 and offered a $1 million prize for each.

Only the Poincaré conjecture has been resolved, by Perelman’s work in 2002–2003; the other six remain open.

image

Smale’s problems: one AI-assisted resolution

Smale’s 1998 list of eighteen “problems for the next century” is a modern successor to Hilbert’s list.

Five rows have been resolved.

One was resolved with AI help: Alpöge and Claude Fable’s 2026 counterexample to the Jacobian conjecture in dimensions three and above.

image

The Open Problems Project: no clear acceleration

The Open Problems Project is a maintained list of seventy-eight computational-geometry problems, many asking for an algorithm, a complexity bound, or an NP-hardness proof.

Seventeen rows have been resolved: thirteen by 2010, followed by isolated resolutions in 2015, 2019, 2023, and 2024. None is AI-attributed.

The recent pace is slower, not faster, although missing dates for when problems entered the list prevent treating this as a clean fixed-cohort solve rate.

image

Ben Green’s 100 open problems: possible acceleration

Ben Green’s 100 open problems is a working list in additive combinatorics, number theory, discrete geometry, and harmonic analysis, circulated since 2018 and revised by Green himself.

Of 101 scored rows, 13 have dated resolutions from 2019 to 2025, a steady rate of just under two per year. The figure is Green’s December 2025 revision, so later events are not in the bars.

After that revision, Liam Price and GPT-5.4 Pro resolved Problem 44 (also Erdős #1202) in the negative, OpenAI Astra constructed a non-sofic group, answering the sofic half of Problem 100 (the hyperlinear half remains open), and Ma, Tang and Xu resolved Problem 90. Green has not yet marked these headings solved.

image

Discovery in Algorithms

NanoGPT speedrun: AI contributions, acceleration unclear

nanoGPT is a collective competition to minimize the training time for an LLM to reach a specific loss on held-out text, given fixed hardware. We can benchmark progress back to GPT-2, and the overall reduction in training time over 2019-2026 has been estimated at about 700-fold.7

There have been five AI-attributed contributions. Taken at face value, they have contributed a fairly small share of the overall progress. Jerry Tworek says:

“Given how many tokens have been spent on nanogpt speedruns and not much coming out of it yet, we have at least a few nights of good sleep ahead.”

It is difficult to judge whether there has been an acceleration. Progress in 2024 mostly involved catching up with the frontier. There was little progress from January through August 2025, but since September 2025 there has been regular progress (see Manish Shetty’s discussion of the history of nanoGPT here). It is conceivable that the acceleration in late 2025 and early 2026 was due to unattributed AI use.

image

CIFAR-10 speedrun: AI contributions, acceleration unclear

A public competition to reach 94% test accuracy on CIFAR-10 in as little wall-clock time as possible, on a single A100.

The newest acknowledged record is AI-set: Hiverge’s engine cut the previous 2.59-second record by about 23%. A later 1.828-second claim (Fulcrum/Fable) is unacknowledged and has specification-gaming caveats.

The two recent contributions are not outstandingly large by historical standards.

image

Hutter Prize compression: too sparse

A prize for compressing a fixed 1 GB Wikipedia dump, scored on compressed size plus the decompressor, under a single-CPU time and memory cap (no GPUs). The corpus has been frozen since 2006.

No entry claims a language model wrote the compressor. The leaderboard has not moved since October 2023.

image

Gurobi mixed-integer programming: no acceleration

Gurobi reruns each new solver release on the same machine and the same model set, so the series is a vendor-reported speedup with hardware held fixed.

Four annual releases (2022–2025) gained 13%, 8.6%, 13.1%, and 0.6%, a cumulative factor of about 1.4 since version 9.5. None of the announcements credits AI.

The release is only annual, so we may see big speedups in 2026.

The current pace of improvement seems to be slower than previously. Grace (2013) says “some mixed integer programming (MIP) algorithms, run on modern MIP instances with modern hardware, have roughly doubled in speed each year”.

![image](https://metr.org/