Berkeley AI Research、LLM の信念更新手法を公開し長期対話効率化を目指す

*Overview of ABBEL compared to traditional recursive summarization. Beliefs replace the full interaction history as the agent’s working context, and belief grading improves performance by

supervising the contents of each belief state..*

As task horizons grow, LLM contexts can’t scale forever. Self-summarization enables concise, interpretable contexts, but at a significant performance cost, especially for human assistance domains where high quality data is scarce, e.g., collaborative code generation. We address this with ABBEL: a framework that isolates and supervises the information content of summaries in the form of natural-language belief states.

Motivation: the cost of recursive summarization

For language models to effectively assist with increasingly complex tasks such as software development, they must be able to interact with us over hundreds or even thousands of steps. For such long tasks, it is impractical to keep the history of the entire interaction in context. The heuristic approach used so far has been summary generation, sometimes called context compaction. For example, Cursor’s latest model composer 2.5 uses compaction during training for improved performance (Cassano et al., 2026). Alongside composer, Grandcode (DeepReinforce et al., 2026), the first system to consistently beat all human competitors in online coding competitions, despite using one of the newest efficient attention models (Qwen 3.5-397B),1 still found it necessary to employ context summarization.

But compaction has a problem. Despite seemingly low performance gaps in benchmarks, model servers like Cursor continue to recommend that users avoid compaction with their coding assistants in the middle of a task (Heule et al., 2026).

To understand why, see below the performance over RL fine-tuning of a Context summary model compared to full context models in Combination Lock, a Wordle-like game that allows up to 16 guesses.2 Though both model types improve over the course of training, the summary model never closes the gap.

*Fig. 1: Average attempts to guess the target word on Combination Lock over RL fine-tuning (lower is better). Context-summary policies improve with training but do not close the gap to full-context policies.*

Making models self-summarize while completing a task increases the complexity of the learning problem. While this could typically be addressed by training with more data, the performance degradation observed in real world interactive settings likely arises from the difficulty we have in creating and using human simulators effectively to generate high quality training environments (Lin et al., 2025, Tomlin et al., 2025). Thus, the better you can learn to summarize on the limited and messy multiturn interaction trajectories you can collect, the better off your model will be for downstream users.

ABBEL: acting through belief bottlenecks

*Fig. 2: Autoencoder-inspired belief grading. The model encodes prior belief, action and observation (bt, at, ot) into posterior belief bt+1 and is rewarded for how well select information from the history can be reconstructed from that belief.*

To address poor learning efficiency, we isolate the summary generation task. Drawing inspiration from recursive Bayesian estimation, we formulate summaries as belief states, which we periodically prompt the model to update based on new information.3

‹ Prev

Pause

Next ›

1 / 16

*Fig. 3: ABBEL rollout. Belief updates from the latest observation alternate with action selection conditioned only on the current posterior belief.*

Belief grading

We then extract and supervise the contents of the belief states (Fig. 2, Belief Grading). Belief grading can be thought of as adding an auxiliary RL task, using heuristics designed to capture what makes a good belief as the reward. An example heuristic for coding could be shorter is better, but closer to being able to reconstruct the git diff is also better, so balancing these would yield a good belief. In domains where good heuristics are hard to define, we propose a general autoencoding-inspired grading function, which treats the current language model πθ as both encoder and decoder of information from

the history, and the belief states as the codes. We grade each belief bt+1 by how well it can be used by the current model πθ to reconstruct the most recent observation ot:

*Eq. 1: Reconstruction grading objective. Here bt+1 is the updated belief, ot the latest observation, at the action just taken, bt the prior belief, pI the task prompt, and πθ the current model. Higher grades reward beliefs that retain information needed to decode the latest observation.*

What do we gain by grading beliefs?

Collaborative coding on CollabBench

We demonstrate the utility of belief grading in our motivating domain of human-driven assistive coding, with the CollabBench environment from Sweet-RL (Zhou et al., 2025).

*Fig. 4: CollabBench collaborative coding environment. The agent asks clarifying questions, then submits a function scored against hidden unit tests.*

We see that with the general reconstruction-based belief grading function we reduce the performance gap from full context models by about 50%, and train in 50% fewer steps compared to training models to summarize without belief grading (no BG). After training, ABBEL still uses significantly less memory than the full context setting, as measured by the peak context token length (Peak Tokens).

Model

Test Pass Rate ↑

Success Rate ↑

Peak Tokens × 10² ↓

Training Steps ↓

Full Context

0.52±0.02

0.39±0.02

14.08±0.55

100

ABBEL (no BG)

0.46±0.02

0.31±0.02

4.20±0.37

100

ABBEL-rec-BG

0.48±0.01

0.36±0.01

6.01±0.33

50

*Fig. 5: CollabBench results. With reconstruction belief grading, ABBEL-rec-BG recovers about half the gap to full context while using fewer peak tokens, and trains in 50 steps instead of 100.*

Combination Lock

Additionally, in CombinationLock, we demonstrate that ABBEL with a belief grader which leverages domain knowledge (by computing useful statistics over the history and checking that they can be reconstructed from the belief state), enables even higher learning efficiency than full context (FULL CTX) models.

*Fig. 6: Average attempts to guess the target word on Combination Lock (lower is better). With domain-knowledge belief grading, ABBEL approaches or exceeds FULL CTX in this setting; without belief grading, learning is slower.*

Multi-objective question answering

In a third environment, multi-objective question answering (from MEM1 Zhang et al., 2025, a recent work which performed end-to-end optimization in a modified version of typical recursive summarization), we demonstrate the utility of isolating belief states from reasoning, by showing that a Peak Belief length Penalty (more details in paper) significantly reduces memory usage with minimal performance degradation, unlike is commonly observed when penalizing reasoning lengths (Arora et al., 2025).