Context squeezed to 8K: Opus 4.5 score holds, but open-source models collapse

NVIDIA released a paper with a very direct title: You don't need long context. The authors include Peter Belcak, Pavlo Molchanov, and others; arXiv:2611.05306. Elvis highly recommended it in a tweet and posted several sets of numbers.

The paper discusses whether agents should frequently compact. The approach is not complicated: at each step, don't carry the full transcript, only a short state summary. DAIR.AI's paper page breaks the conclusion into five points; the core is that short context did not break long-task performance.

Look at the data first. Claude Opus 4.5 takes at most one action per step and scores 70.2 on SWE-bench Verified, versus a full-history baseline of 70.0. Median peak context drops from 32K tokens to 8K. In an even simpler case: doing compaction once at the end of the conversation, Opus 4.5 goes from 70.0 to 74.8. With shorter context, the score didn't drop; it rose.

The cost math is also clear. The paper says later tokens are more expensive to generate, with the gap reaching up to 7x on frontier open-weight models. debugmodeon added the mechanism: every decode step must read the entire KV cache; the shorter the context, the cheaper each token is, and the same GPU can fit more agent runs. 70.2 vs. 70.0, with peak context only one quarter as large—not a bad trade.

But the title can easily mislead. The paper does not say all models can be compressed freely. J.W pointed out that under the tightest compression, Qwen 3 Coder Next drops from 53.0% to 3.6%. The paper also sees the pattern hold on Terminal-Bench, FRAMES, BFCLv3, and multilingual SWE-bench, while acknowledging small losses in some settings. GPT 5.6 Sol lost 2.0 and 3.3 points under single-action compression. Open-source models and closed-source frontier models have different tolerance for compression.

There are other pitfalls in real agents. Luca Iaconelli worries that decisions made early in a session get lost after several rounds of compaction. k11i mentions a multi-agent scenario: agent B receives agent A's compressed summary and treats it as ground truth—that's when problems arise. Compression itself is not bad; what's bad is treating the compressed state as complete fact.

So this paper is more like a reminder to test. Elvis says he doesn't quite trust built-in auto-compaction; he built his own, optimized for the task, and evaluates frequently. That attitude is more practical than the conclusion. Before a million-token agent run, try aggressive compression on your own task. Look at peak context, look at acceptance results, and see whether the lost details affect decisions. If you can still finish the work with context squeezed to 8K, then talk about saving money.

Paper: https://arxiv.org/abs/2611.05306
PDF: https://arxiv.org/pdf/2611.05306
DAIR.AI paper page: https://academy.dair.ai/papers/you-dont-need-long-context-2611.05306
Tweet: https://x.com/omarsar0/status/2117956281910829083