Post-Training Is Eating AI's Compute Budget
Pretraining has fallen from 67% to 7%, while post-training has risen from 5% to 55%. At Cornell, Nathan Lambert laid out the evolution of post-training and six open problems that no one has solved.
Nathan Lambert gave a talk at Cornell University—51 slides—that traced the evolution of post-training over the past few years. Uncle Moon reposted it, commenting, “Really well done; the development of post-training is organized clearly.”
Post-training refers to the set of techniques that make pretrained models more useful for downstream tasks: instruction fine-tuning (SFT), preference alignment from human feedback (RLHF), and reinforcement learning with verifiable rewards (RLVR). The field is changing so fast that Lambert summed it up in one sentence: post-training recipes have changed more in the past year than in the previous three years combined.
Most notable is the shift in compute allocation. SemiAnalysis data shows that across the combined compute of Anthropic and OpenAI, pretraining fell from 67% in Q1 2024 to a projected 7% in Q4 2026; post-training and RL rose from 5% to 55%.

For individual models, the gap is just as striking. DeepSeek R1’s RL compute was about 5% of its pretraining compute. In 2026, Beam is over 1.7x, and Inkling 2 Small about 4x. Post-training has gone from a rounding error relative to pretraining to the main event.
Methods are also iterating rapidly. In 2022–2023, InstructGPT’s three-stage recipe was standard: SFT, reward modeling, and PPO. In 2024, Llama 3 and Tülu 3 added more stages and RLVR on top of it. In 2025, DeepSeek R1 made large-scale RL central. In 2026, MiMo-V2-Flash introduced MOPD, training multiple domain-expert models and then distilling them back into a final model.

Lambert’s assessment is that full post-training is becoming like pretraining in early 2023: the resource barrier is rising fast. A new division of labor is therefore emerging: many people start directly from off-the-shelf reasoning models and do domain adaptation. He calls this post-post-training. Harvey post-trained a legal model from Kimi K3’s base model; Mercor and SkyRL post-trained Qwen3.5-397B into a knowledge-work agent and released both scripts and weights.
The main body of the talk covers six open problems he considers most important, ordered from low to high compute requirements:
Harness specialization and design. How to design environments and context management so models work efficiently on specific tasks. His example is Context Language Models, which let the model directly manage its own context rather than relying on structured harness commands.
Character training. How to shape a model’s personality and values. Industry has largely rebuilt this direction—OpenAI released a new Model Spec for agents—but the open-source literature has not caught up. Lambert says this was one of Trillium Labs’ earliest areas of investment, and measuring performance is extremely difficult.
Open-source RL environments. Post-training lacks the kind of shared data commons that pretraining had; solving this requires investment from individuals and institutions. Good data projects often end with the lead author rereading every data point. TMax is one example: it builds terminal-bench data and post-trains Qwen models from 2B to 27B into terminal agents, reaching SOTA.
Inference efficiency in RL. Why hasn’t any paper clearly shown how to achieve the massive token-efficiency gains that frontier labs seem to get? One natural idea: spend more RL compute on the hardest problems. GRPO generates multiple completions per prompt; if all are correct there’s no gradient, and if all are wrong it gives up. NGU’s approach: if all completions are wrong, continue sampling with probability p.

RL scaling laws. How to design experimental methods for large-scale post-training. Pretraining has the concept of a ladder; RL has no corresponding thing.
Multi-agent scaling laws. Multiple agents in parallel can trade for wall-clock time and can also improve peak performance on hard tasks. But which tasks benefit, how to use model diversity, and whether training methods need to change for multi-agent systems all remain unanswered.
The slides close with a discussion of RSI (recursive self-improvement). Anthropic’s R&D automation index shows Claude-led R&D task share rising from less than 1% in February 2026 to 26% in August; fully automated is still zero. Lambert says LLMs accelerate output but hardly accelerate understanding. He himself barely writes code anymore, yet as a scientist he is busier than ever. Young researchers, by contrast, have to build understanding through inefficient manual work.

One commenter mentioned a long post Ramez Naam published that day, arguing that the classic charts showing AI task durations doubling differ enormously from the real internal data at OpenAI and Google—and the gap is still widening. Lewis Tunstall commented that this kind of analysis helps hedge against the scary charts on X.
Another reply pointed to “AI slop”—a commenter said they’d like to see more research on how models produce first-hand, dishearteningly bad writing. Zhizhuo Zhou is working on a related benchmark.
Cornell student Anmol Kabra presented his paper PhantomEnvironments: training LLMs to do multi-hop search in purely rule-generated fictional worlds, without relying on LLM-generated data, at zero marginal cost; the trained agents transfer to real-world benchmarks. This is an early prototype of a solution for open-source RL environments.
Lambert says the talk video will be re-recorded. The full slide deck is available at natolambert.com.
发布时间: 2026-11-23 03:25