SWE-1.7：コストの大幅削減で最先端知能を実現（22 分読）

Ben Pan*, Carlo Baronio*, Rohan Choudhury, Eric Lu, Ryan Kim, Deniz Birlikci, TC Qin, Sam Lee, Fermi Ma, Allen Liu, Yang Liu, Sampriti Panda, Jacob Teo, Ray Wang, Gary Chang, Steven Cao, Silas Alberti*equal contribution

07.08.26

Today, we’re launching SWE-1.7, the most capable model we’ve trained so far. It reaches frontier-level intelligence at a much lower cost, advancing the cost-performance Pareto curve.

SWE-1.7 is the result of broad improvements across our RL pipeline: better infrastructure, more stable training, higher-quality data, and new techniques for long-horizon tasks. Since SWE-1.7 was trained from a Kimi K2.7 base, which had already undergone extensive RL post-training, the large additional gains from our own training challenge the idea of a ‘post-training ceiling’ and suggest that RL can push capabilities much further than previously believed.

At Cognition, we have been formulating and refining principles for good agentic software engineering both in evaluation, with FrontierCode1,2, and now in training, with SWE-1.7. Our model is particularly optimized for longer-horizon asynchronous tasks, an important component of high-quality software engineering.

SWE-1.7 is available today in Devin (Web, Desktop, and CLI) via Cerebras at 1000 TPS. We encourage you to try it for yourself!

The rest of this post covers how we trained SWE-1.7: the infrastructure, algorithms, and data work behind our model. We cover four important components that stand out.

• Preserving entropy and stabilizing training: Long RL runs face two challenging problems: entropy collapse, and instability due to numerical drift between training and inference. We hunted down and addressed causes of each, which enabled training to keep improving well past where earlier runs stalled.
• Multi-cluster training and fault tolerance: RL doesn’t need all of its inference compute in one cluster. We trained on clusters across three continents, shipped weight updates through object storage, and built fault tolerance so that hardware failures never stalled the run.
• Curating high-quality data: We built an extensive data-quality pipeline that runs each task through automated execution tests, filters out tasks with low learning signal, and hardens tasks to prevent reward-hacking.
• Self-compaction for long-horizon tasks: The model learns to summarize its working state and resume from the summary, extending task horizons past the raw context window. We use an alternating length penalty to incentivize concise output without sacrificing correctness.

Finally, we conclude by sharing some observations on interesting behavioral tendencies, such as careful exploration and concise reasoning, that the model acquired as a result of our training setup.

Preserving Entropy and Stabilizing Training

We found training stability to be a key contributor to predictable improvement at scale.

When training with asynchronous RL3, one of the most problematic issues we encountered was the KL divergence mismatch between inference and training4, since the trainer policy is usually different from the sampling policy. In the past, to correct for this (albeit at smaller scale), we used importance-sampling5 and quantization-aware training for low-precision rollouts in NVFP4 + experts routing replay6,7.

Here we present additional interventions that become more important at larger scale.

We find that top-p sampling8 contributes significantly to staving off entropy collapse9,10, where a strong model stops exploring and reward plateaus within a few hundred steps.

Very low probability tokens are often part of trajectories that have gone off track or out of distribution. These trajectories are likely to produce low reward, and properties of the softmax function lead to these tokens sharpening the token probability distribution. Indeed, suppose we have three tokens with logits x1>x2≫x3x_1>x_2\gg x_3 and probabilities pi=exiex1+ex2+ex3p_i = \frac{e^{x_i}}{e^{x_1} + e^{x_2} + e^{x_3}}, where token 3 is a low probability token that leads to low reward. If we sample token 3, the gradient of its logprob with respect to the logits x1,x2,x3x_1, x_2, x_3 is:

∇log⁡p3=∇log⁡[ex3ex1+ex2+ex3]=[−p1−p2p1+p2]\nabla \log p_3=\nabla\log \left[\frac{e^{x_3}}{e^{x_1}+e^{x_2}+e^{x_3}}\right]= \begin{bmatrix} -p_1 \\ -p_2 \\ p_1+p_2 \end{bmatrix}

and the policy gradient update to the logits is Δxi∝A^ ∇log⁡p3\Delta x_i \propto \hat{A}\ \nabla \log p_3, where A^\hat{A} is the advantage of the sampled token. Since this trajectory earns low reward, A^<0\hat{A} < 0 and the updates are

Δx1∝∣A^∣ p1,Δx2∝∣A^∣ p2,Δx3∝−∣A^∣ (p1+p2).\Delta x_1 \propto |\hat{A}| \, p_1, \qquad \Delta x_2 \propto |\hat{A}| \, p_2, \qquad \Delta x_3 \propto -|\hat{A}| \, (p_1+p_2).

In these updates, x3x_3 is penalized, and x1x_1 grows more than x2x_2. Sampling x3x_3 therefore widens the lead of the already-dominant token, sharpening the distribution and decreasing entropy. Top-pp sampling prevents these low probability tokens from being sampled and used as optimization targets in the first place!

This entropy-preservation effect makes top-pp sampling desirable in our rollouts. But naively implementing top-pp clearly increases the training-inference mismatch — the trainer computes probabilities as a selection from all tokens, while rollouts sample from the top-pp subset, so the distributions have higher divergence, leading to collapse after a small number of steps. Thus, we implement sampling distribution replay11, where we record a kept-set of tokens available for sampling at rollout time, and renormalize probabilities with those masks in the trainer. With this fix, our run’s entropy stays roughly constant over the course of training and inference-training divergence stays bounded.

Another interesting result of using top-p sampling replay is a targeting of only tokens with p<top_p_thresholdp < \text{top\_p\_threshold}. Tokens with probability above the threshold have a keepset of size 1, so their renormalized probability distribution is a constant 1, and gradients are zeroed out. We found empirically that a large fraction of the tokens sampled by the model are above standard top-p thresholds, so they are excluded from the overall gradient computation. This reduces gradient noise and lets the optimization algorithm focus on the high-learning signal tokens in the trajectory.

We also find benefits from using the Muon optimizer12 and eliminating non-deterministic operations in the trainer.

Multi-cluster Training

Cognition is a fast-growing research lab entering an established landscape that is heavily compute-constrained. We aim to train trillion-parameter models, but today, large clusters with 10-100k chips on a single network fabric are a scarce resource. In contrast, smaller clusters around the world are abundant, if used together correctly.

In this setting, the structure of RL works in our favor. RL decomposes naturally across multiple clusters. Only the trainer must live on a single high-bandwidth cluster. The inference engines that generate rollouts are self-contained. They can run anywhere and need nothing but the current weights.

We invested in infrastructure that makes use of this property. Our RL training spans four datacenters across three continents, combining our own GPUs across multiple clusters with additional compute from inference providers like Fireworks. The result is that we can scale RL training far beyond what any single cluster would allow.

Data Quality

Data is the core determining factor in what capabilities and skills our model learns. As such, we ensured that the data we trained on was calibrated and sufficiently difficult, and disincentivized undesirable behaviors to keep the model well-aligned. We focused most on the following aspects:

• Verifier quality: A task’s verifier can be wrong in two directions: accepting incorrect solutions (false positives) or rejecting valid ones (false negatives). We devised extensive quality-assurance pipelines to minimize observations of false positives and false negatives in our training.
• Difficulty: On tasks where the model always solves or always fails, we don’t observe any meaningful learning signal. Instead, we curated training data that the model only solves a low fraction of the time, which generates real learning signal while at the same time pushes model intelligence.
• Cheating detection and prevention: We employed a variety of defenses against different forms of cheating. For instance, we network-restricted our sandboxes and stripped them of git history and reference artifacts. We also isolated the grading path from the agent itself. In addition, we employed programmatic checks to catch known exploit signatures. Finally, to ensure proper incentives, we assigned reward 0 to trajectories with any instance of cheating attempts, regardless of whether they succeeded.

Results: Model Behaviors

Due to extensive RL, SWE-1.7 exhibits noticeably different behavior from Kimi K2.7 Code, its base model. Firstly, it is significantly more aligned and trustworthy than K2.7 or other frontier open-source models. We expand on this extensively in our companion blog post, *Measuring the Trustworthiness of Open-Source-Derived Models*.

One behavioral difference we noticed in SWE-1.7 is condensed chain-of-thought. Compared to Kimi-K2.7-Code, SWE-1.7’s first chain-of-thought has a much lower function-word ratio (fraction of words that serve as grammatical “glue”) and nearly half the average number of words per sentence. We think this was influenced directly by the budget phases in our alternating length penalty. We’ve included a couple of examples of condensed chain-of-thought in the dropdown below.

The other major behavioral difference we observed is that SWE-1.7 explores the codebase much more thoroughly before acting, as you can see in the number of tool calls, file reads, and searches the model executes.

This shows up most clearly in bug-fixes. A bug report typically describes one primary symptom, but the underlying issue often affects a larger surface area. SWE-1.7 is much more likely to investigate the root cause of the bug and consider edge cases, hypotheticals, adversarial inputs, and beyond-the-ask requirements than Kimi-K2.7-Code. Through its enhanced codebase exploration, SWE-1.7 also does a much better job understanding the exact design decisions that need to be made. In addition, we’ve observed that SWE-1.7 tends to settle ambiguous semantics by experimenting and probing, for example by writing small Python scripts, rather than guessing.

We believe these behaviors arise directly from the extensive quality-assurance measures we took to strip out false positives and false negatives in our data, forcing our model to come up with more complete, end-to-end solutions. We think that SWE-1.7’s increased due-diligence directly translates to higher performance on various benchmarks. We’ve attached a couple of example trajectories in the dropdown below.

The extra thinking comes at a small cost in increased change scope. As described in FrontierCode1, a good solution modifies only the minimal set of files needed, without touching unrelated code or introducing unnecessary refactors. Since SWE-1.7 reasons more, it also does more: writing additional test cases and touching more files than the task naively requires. We’ve noticed this trend consistently in models across the industry: as reasoning increases, the scope of files that the model touches also expands. This is an axis we’re excited to improve on.

Evaluation Methodology

• All models are evaluated under their maximum reasoning effort.
• Terminal-Bench 2.117: we evaluate on our own internal evaluation framework, using Claude Code for Anthropic models, Codex for OpenAI models, and Devin CLI for other models, with timeout=4h.
• SWE-Bench Multilingual18: we use self-reported numbers when available and evaluate on Devin CLI otherwise.
• FrontierCode 1.1: see our blog post2.

References

• [1]E. Lu, B. Pan, D. Birlikci, S. Lee, R. Wang, R. Choudhury, F. Ma, TC Qin, C. Baronio, and S. Alberti, "Introducing FrontierCode," June 2026. cognition.com/blog/frontier-code
• [2]E. Lu, B. Pan, F. Ma, A. Lombardi, D. Birlikci, S. Lee, R. Wang, R. Choudhury, TC Qin, C. Baronio, J. Teo, J. H. Lee, and S. Alberti, "FrontierCode 1.1," July 7, 2026. cognition.com/blog/frontier-code-1.1
• [3]A. Piché et al., "PipelineRL: Faster On-policy Reinforcement Learning for Long Sequence Generation," arXiv:2509.19128, 2025.
• [4]F. Yao et al., "Your Efficient RL Framework Secretly Brings You Off-Policy RL Training," 2025. fengyao.notion.site/off-policy-rl
• [5]B. Pan, C. Baronio, A. Tam, P. Marsella, M. Jain, D. Chiu, Swyx, and S. Alberti, "Introducing SWE-grep and SWE-grep-mini: RL for Multi-Turn, Fast Context Retrieval," 2025. cognition.com/blog/swe-grep
• [6]"Stabilizing MoE Reinforcement Learning by Aligning Training and Inference Routers" (Rollout Routing Replay, R3), arXiv:2510.11370, 2025.
• [7]C. Baronio, B. Pan, S. Lee, E. Lu, S. Cao, R. Choudhury, A. Zweiger, R. Wang, G. Chang, and S. Alberti, "An Early Preview of SWE-1.6 and Research Update," March 2026. cognition.com/blog/swe-1-6-preview
• [8]A. Holtzman et al., "The Curious Case of Neural Text Degeneration," arXiv:1904.09751, 2019.
• [9]G. Cui et al., "The Entropy Mechanism of Reinforcement Learning for Reasoning Language Models," arXiv:2505.22617, 2025. arxiv.org/abs/2505.22617
• [10]S. Yu et al., "DAPO: An Open-Source LLM Reinforcement Learning System at Scale," arXiv:2503.14476, 2025. arxiv.org/abs/2503.14476
• [11]DeepSeek-AI, "DeepSeek-V3.2: Pushing the Frontier of Open Large Language Models," arXiv:2512.02556, 2025. arxiv.org/abs/2512.02556
• [12]K. Jordan et al., "Muon: An optimizer for hidden layers in neural networks," 2024. kellerjordan.github.io/posts/muon — see also J. Liu et al., "Muon is Scalable for LLM Training," arXiv:2502.16982, 2025.
• [13]Fireworks AI, "Frontier RL Is Cheaper Than You Think," 2026. fireworks.ai/blog/frontier-rl-is-cheaper-than-you-think
• [14]DeepSeek-AI, "DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning," arXiv:2501.12948, 2025. arxiv.org/abs/2501.12948
• [15]C. Baronio, P. Marsella, B. Pan, and S. Alberti, "Kevin-32B: Multi-Turn RL for Writing CUDA Kernels," 2025. cognition.com/blog/kevin-32b
• [16]Kimi Team, "Kimi K2.5: Visual agentic intelligence," arXiv:2602.02276, Feb. 2026. arxiv.org/abs/2602.02276
• [17]The Terminal-Bench Team, "Terminal-Bench: A Benchmark for AI Agents in Terminal Environments," 2025. tbench.ai
• [18]J. Yang et al., "SWE-smith: Scaling Data for Software Engineering Agents," arXiv:2504.21798, 2025. arxiv.org/abs/2504.21798 (SWE-bench Multilingual: swebench.com/multilingual)