Kimi K3 の能力と課題を分析

Kimi K3 is a very good model with excellent benchmarks. Assuming its weights are released as planned it will become, purely in terms of raw capability, the strongest open model.

Do not get carried away. Do not judge Kimi K3 only its relative strengths. In aggregate it is several months behind the closed model frontier, at least four and my median guess is six, with the post-training closer and the pre-training farther out. This is less months than before, but the months are denser now.

It is somewhat distilled. It likely outperforms on benchmarks relative to practical performance. All its benchmarks are scored at maximum effort, typically a lot more tokens than are used in similar tests by Fable or Sol. Performance looks jagged. Kimi will be excellent at some things, less so at other things.

We will know more over the coming weeks. For now access is spotty and not that many people have actually had the chance to try Kimi K3, so I have larger error bars than usual around its capabilities. Alas, time waits for no one, so we press on.

It is the largest open model so far at 2.8T, on the upper end of possible sizes for Claude Opus and near the bottom of possible sizes for Mythos, which explains many of its gains. It is slow and appears hungry for tokens. A lot of the positive reactions are to this being a big model, which thus has at least a decent amount of ‘big model smell’ and generally trades being slower and more expensive for some performance gains. That’s a great move, but it should be a while before they can do it again.

Distillation from Claude is clearly part of the story, likely largely from Fable, and is clearly nothing close to the whole story. Clearly Moonshot would do this even if it helped only a little. The timing of them releasing a bigger model is suggestive.

It is again a good model, but once you correct for the overperformance on benchmarks and look at expected practical performance, it is not clear this is so different from what you would have expected from a Kimi K3 that had 2.8T parameters.

Consider that Kimi K3’s (preliminary unofficial) Epoch Capabilities Index is exactly on the Chinese trend line.

Kimi K3 is absolutely worth checking to see if it fits into your workflows. At this price point, for both the API and the subscription, it is not going to fill the role of the smaller cheaper open models, and I expect it to usually lose out in a fight with the top closed models, but there are going to be some places where Kimi K3 is a good choice.

Andrew Curran: Following the success of Kimi K3, Moonshot AI has informed investors that it plans an IPO in Hong Kong within the next six months, according to Bloomberg.

Good idea. Strike while the iron is hot.

Table of Contents

DeepSeek Moments: Here We Go Again.

We Had a Moment (Reprise from June 2025).

The Story Since Then.

The Kimi K3 Announcement, Pitch and Basic Facts.

On Modern Benchmaxxing.

Other People’s Benchmarks.

Benchmarks Are Not The Real World.

Technical Safeguards? What Are Those?

Things Kimi Can Do.

Things Kimi Cannot Do.

Things It Is Not Easy To Get Kimi To Do.

Open Weight Models Are Unsafe And Nothing Can Fix This.

Dean Ball Attempts To Be Constructive.

OpenAI Employees Are Relatively Bullish On This One.

Kimi K3 Is Relatively Strongest At Typical Agentic Coding and 3D.

Reactions.

Who Are You?

How Did They Do It?

Conclusion.

DeepSeek Moments: Here We Go Again

All discourse about Chinese models lives in the shadow of the DeepSeek moment.

There are a lot of people who really, really want another DeepSeek moment to happen.

These people really, really want to tell the story that Chinese open models are catching up to American closed models, that AI and inference will become commoditized.

Their motivations vary. They often want to affirm open models, or the importance of the ‘tech stack.’ Others simply want to see OpenAI and Anthropic go down, or know that such claims sell. Often the ultimate objective is to argue against all AI regulations, or anything that might ‘slow us down’ or cause us to ‘lose to China.’

It is actively suicidal to respond to ‘the Chinese have better models now’ with ‘then we had better sell them the compute so they can run them and also build even better ones.’ Yet every time, yes, people will argue that. Sigh.

Often they simply want to tell American AI to stop taking precautions, to stop being annoying and take down the classifiers, as in ‘genie is out of the bottle, so release the bigger genie with unlimited wishes, it’s the only way.’ People really would take major catastrophic risks rather than deal with classifiers, and are Big Mad about this.

Google was down 4.4% on the day, SpaceX was down 3.1% and Nvidia down over 2%, and tech stocks were down again on Friday, so plausibly we’re doing this again.

And yep, we are at risk of doing this again:

Axios (being wrong): China just erased America's AI lead

Seán Ó hÉigeartaigh: No it didn't. (although Kimi K3 is undoubtedly impressive)

The pattern is:

Chinese model releases.

There is some impressive benchmark cited.

Therefore, America’s lead is gone, QED, that’s it, no, really, that’s it.

In the Axios case the benchmark in question is Arena. Based on that alone, they state as fact that America’s lead is gone.

I would ignore, but this style of logic has convinced a lot of Washington D.C. multiple times, and that has had substantial policy impact.

I shudder to think what such folks might do now that they also know about Mythos. The confusion over Fable jailbreaks could easily extend to a broader dumb panic.

The original DeepSeek moment happened because of a confluence of events.

Let’s review.

We Had a Moment (Reprise from June 2025)

We all remember The DeepSeek Moment, which led to Panic at the App Store, lots of stock market turmoil that made remarkably little fundamental sense and that has been borne out as rather silly, a very intense week and a conclusion to not panic after all.

Over several months, a clear picture emerged of (most of) what happened: A confluence of narrative factors transformed DeepSeek’s r1 from an impressive but not terribly surprising model worth updating on into a shot heard round the world, despite the lack of direct ‘fanfare.’

In particular, these all worked together to cause this effect:

The ‘six million dollar model’ narrative. People equated v3’s marginal compute costs with the overall budget of American labs like OpenAI and Anthropic. This is like saying DeepSeek spent a lot less on apples than OpenAI spent on food. When making an apples-to-apples comparison, DeepSeek spent less, but the difference was far less stark.

DeepSeek simultaneously released an app that was free with a remarkably clean design and visible chain-of-thought (CoT). DeepSeek was fast following, so they had no reason to hide the CoT. Comparisons only compared DeepSeek’s top use cases to the same use cases elsewhere, ignoring the features and use cases DeepSeek lacked or did poorly on. So if you wanted to do first-day free querying, you got what was at the time a unique and viral experience. This forced other labs to also show CoT and accelerate release of various models and features.

It takes a while to know how good a model really is, and the different style and visible CoT and excitement made people think r1 was better than it was.

The timing was impeccable. DeepSeek got in right before a series of other model releases. Within two weeks it was very clear that American labs remained ahead. This was the peak of a DeepSeek cycle and the low point in others cycles.

The timing was also impeccable in terms of the technology. This was very early days of RL scaling, such that the training process could still be done cheaply. DeepSeek did a great job extracting the most from its chips, but they are likely going to have increasing trouble with its compute disadvantage going forwards.

DeepSeek leveraged the whole ‘what even is safety testing’ and fast following angles, shipping as quickly as possible to irrevocably release its new model the moment it was at all viable to do so, making it look relatively farther along and less behind than they were. Teortaxes notes that the R1 paper pointed out a bunch of things that needed fixing but that DeepSeek did not have time to fix back then, and that R1-0528 fixes them, and which weren’t ‘counted’ during the panic.

DeepSeek got the whole ‘momentum’ argument going. China had previously been much farther behind in terms of released models, DeepSeek was now less behind (and some even said was ahead), and people thought ‘oh that means soon they’ll be ahead.’ Whereas no, you can’t assume that, and also moving from a follower to a leader is a big leap.

There was highly related to a widespread demand for a ‘China caught up to the USA’ narrative, from China fans and also from China hawks of all sorts. Going forward, we are left with a ‘missile gap’ style story.

There are also a lot of people always pushing the ‘open models win’ argument, and who think that non-open models are some combination of doomed and don’t count. These people are very vocal, and vibes are a weapon of choice, and some have close ties to the Trump administration.

The stock market was highly lacking in situational awareness, so they considered this release much bigger news than it was, and it caused various people to ‘wake up’ to things that were already known and anticipate others waking up, and there was widespread misunderstanding of how any of the underlying dynamics worked, including Jevon’s Paradox and also that if you want to run r1 you go out and buy more chips, including Nvidia chips. It is also possible that a lot of the DeepSeek stock market reaction was actually about insider trading of Trump policy announcements. Essentially: The Efficient Market Hypothesis Is False.

The Story Since Then

Since then, the idea that China had caught up, or was catching up, kept coming up, drove much discussion around Washington, as echoes of this one moment.

This is then renewed every time a new strongest or exciting Chinese model comes out. Every day that China does not release a model, they look one day farther behind. When they do release, they ‘catch up’ and look less behind.

The top 10 such potential moments since r1 and before K3 were likely these:

Manus.

DeepSeek r1-0528.

Kimi K2.

GPT-5 (in reverse) which spooked a lot of people in highly stupid ways.

DeepSeek v3.1.

Kimi K2 Thinking.

DeepSeek v3.2.

Kimi K2.5.

DeepSeek v4.

GLM-5.2.

Mostly it’s been Kimi and DeepSeek. Manus got a hype train going and spooked people, and GLM-5.2 was by far their strongest offering, putting GLMs on the map.

Many of these were good models, but none fundamentally changed the game.

Roughly, since the DeepSeek moment, when DeepSeek was roughly eight months behind but had matched some key aspects faster via fast following, we have bounced around. For a while it looked like China was quite a lot behind.

GLM-5.2 and Kimi K3 have been impressive. The current best estimate of the time gap is at its lowest point. Events in AI have accelerated all around, so it is not clear that the gap is fewer product cycles than before, and I half expect to be doing this again next week for Qwen. Kimi K3 is 2.8T and seems to still be solidly behind Mythos Preview, which was announced on April 7, so that provides a starting point lower bound of a three month gap.

Ethan Mollick: Kimi is, as I have been saying, a very good model. But it is not a DeepSeek r1 moment, in that it is roughly where I would expect on the curve rather than an unexpected leap. It will be treated as a DeepSeek moment for a variety of reasons especially as more people hear about it.

Ryan Greenblatt, despite being pleasantly surprised by Kimi K3, estimates that the pretrain quality is about halfway between Opus 4 and Opus 4.5 based on forward pass math, but with some other advantages, so ~8 months behind, as one might expect.

The post-training is closer, at least in part because of distillation. Moonshot is clearly innovating, but it is also clearly distilling, both directly and also fast following via looking at outputs and copying techniques.

UK AISI issued a report on everything prior to Kimi K3, showing the time gap for narrow cyber tasks narrowing somewhat over time. Their full report is here.

My understanding is that narrow and relatively easy coding tasks are where open weights model are at their relative strongest, and the benchmark here is approaching saturation.

Indeed, when you look at the full post, you get a different answer for longer tasks.

UK AI Security Institute: On TLO, GLM-5.2 reaches as far as Opus 4.5, a model released less than 7 months before it, while DeepSeek’s V4-Pro falls below Sonnet 4.5 (a sub-cyber-frontier model released 7 months before it). These results are broadly consistent across our other cyber ranges. Notably, GLM-5.2 reached step 7 with marginally fewer tokens than any other model on average, tracking Opus 4.6’s trajectory to step 11 before stalling.

Longer tasks are more relevant in terms of both of the most important things to worry about: Automation of AI R&D and cyber attacks.

One might also worry about bio risks, even if that is not as in fashion, and it is a little concerning we don’t see standard testing on that at all for the open models. That needs to be addressed. We do have the score on OpenAI’s GeneBench-Pro via Andrew Ho. This measures judgment under long-horizon ambiguity in computational biology. Kimi K3 exceeded expectations. Mythos have not been tested. Fable refused most requests in the benchmark.

Beating Opus and GPT-5.5 is impressive stuff. No previous open model came close. We are on the verge of doing some f***ing around and thus finding out. This is a place where plausibly not much happens until suddenly quite a lot happens.

The conclusion of ‘open models will catch up to Mythos in general capability including the thing that currently makes it unique’ is indisputable. That is coming, the question is when, and that will establish the effective gap. Given Kimi K3 we should expect this to happen a few months from now.

If we take both ends of UK AISI’s estimates, the pre-Kimi gap was 4-7 months, down from 6-10 months last year, in an area of relative strength. That’s roughly a similar amount of progress gap in absolute terms, and everything is accelerating.

The Kimi K3 Announcement, Pitch and Basic Facts

2.8 Trillion parameters, 16 of 896 areas active at once which implies ~50B active. This is not easy to run locally, and won’t be that cheap.

$3.00/$15.00, modestly cheaper than Opus and Sol.

Subscription plans are $19/$39/$99/$199 per month. The larger buys have some modest advantages and quota scales linearly with price.

1M token context.

API link, Tech blog link.

Training cutoff is reportedly early 2026.

Open weights promised by July 27th.

All benchmarks run under maximum effort settings.

Kimi: Today, we are introducing Kimi K3 — our most capable model. Kimi K3 is a 2.8T-parameter model built on our Kimi Delta Attention and Attention Residuals, with native vision capabilities and a 1-million-token context window. It is the world's first open 3T-class model, designed for frontier intelligence across long-horizon coding, knowledge work, and reasoning.

While its overall performance still trails the most powerful proprietary models, Claude Fable 5 and GPT 5.6 Sol, Kimi K3 demonstrated frontier-level performance across our evaluation suite, consistently outperforming other tested models.

Kimi.ai: Kimi K3 is now live on on http://Kimi.com, Kimi Work, Kimi Code, and the Kimi API. Open Weights by July 27, 2026.

K3 is built on Kimi Delta Attention (KDA) and Attention Residuals (AttnRes), two architectural updates designed to improve how information flows across sequence length and model depth.

We have also scaled up Mixture of Experts (MoE) sparsity, effectively activating 16 out of 896 experts when paired with a Stable LatentMoE framework.

Together with refined training and data recipes, these structural changes yield an approximate 2.5× improvement in overall scaling efficiency compared to K2, allowing the model to convert compute into intelligence more effectively.

As stated above, they claim strong official benchmarks, although short of Sol or Fable.

The ad, which Tyler Cowen called very positive and very good, falls flat to me, and contains zero useful information.

On Modern Benchmaxxing

We used to see rather explicit benchmaxxing. Labs would train on the test, or on the very narrow thing the test would cover, because we had a limited set of known targets. You had to know which labs did this, to what extent, when looking at numbers.

Our benchmarking tech has improved, and now they collectively measure real things, and there are a variety of backups in case you aim too narrowly.

roon (OpenAI): on the subject of benchmaxxing - it seems benchmarking technology has gotten better recently, like most other technology. people are more skeptical and build these things more carefully. there’s also a explosion of downstream coding customers building internal heldout evals

Looking at the gestalt of different benchmarks is also valuable. Everything should be part of a map that fits into a common pattern that reflects the underlying territory.

You can still absolutely benchmaxx without being as explicit as you used to be.

Benchmarks measure some types of abilities rather than others, and measure shallow rather than deep tasks, and exclude many valuable properties or potential liabilities. And some labs focus more on those aspects, or have more success on them, than others.

You can also set effort to maximum for all the tests, which Moonshot did.

Think of the benchmarks as a lower bound. Kimi’s benchmarks prove it is for real, and it could only underperform (or outperform) them by so much. I still expected, and continue to believe, that they modestly overstate Kimi K3’s relative capabilities.

Other People’s Benchmarks

On the closest thing we have to the One True Benchmark, Kimi K3 does well, confirming claims that overall this model has the third highest benchmarks:

![image](https://substackcdn.com/image/fetch/$s_!XsAE!,w_1456,c_limit,f_auto,q_auto:good,fl_pro