Post-training Is Eating Frontier Labs' Compute, and Open Research Still Has Plenty of Open Space
Frontier labs spend 55% of compute on post-training, yet the open literature still has vast gaps. Nathan Lambert's Cornell talk lays out six open problems, post-post-training, and the RSI debate in one go.
Nathan Lambert presented a new set of slides at Cornell titled “Pushing Frontiers in Open Language Model Post-Training.” He is the Executive Director of Trillium Labs, and the substance of this talk is to put the most important open problems in the post-training field on the table.
Two opening judgments:
- Full post-training at the frontier is becoming like pretraining in early 2023: the resource threshold is rising quickly; it is solvable, but requires a lot of money and long-term commitment.
- The open post-training literature has large gaps on key ideas; looking back in five years, these gaps will very likely matter a lot.
“Post-training” refers to the whole set of methods that make a pretrained next-token model more useful for downstream tasks: SFT, preference alignment/RLHF, and reinforcement learning with verifiable rewards (RLVR). Over the past year, the recipes have changed more than in the previous three years: 2022 to 2023 was the InstructGPT-style SFT→RM→RL; 2024 was Llama 3 and Tülu 3 plus RLVR; 2025 was DeepSeek R1 making large-scale RL central; 2026 was MiMo-V2-Flash introducing MOPD, distilling multiple domain-expert models back into a single student model. DeepSeek V4 and Nemotron 3 Ultra are also using it.

Post-training is eating frontier labs’ compute. SemiAnalysis estimates show that, in OpenAI + Anthropic compute, pretraining fell from 67% in 2024 Q1 to 7% in 2026 Q4, while post-training and RL rose from 5% to 55%.

Looking at individual models is even more direct:
| Model | Pretraining | RL | RL/Pretraining |
|---|---|---|---|
| DeepSeek R1 (Jan 2025) | 2.66M H800 GPU-hours | 0.14M H800 GPU-hours | ~5% |
| Beam (Oct 2026) | <4.1M GB300 GPU-hours | ~7.1M GB300 GPU-hours | >1.7× |
| Inkling 2 Small (Nov 2026) | ~0.11M GPU-hours* | ~0.44M GPU-hours | ~4×* |
*Inkling’s pretraining GPU count is an estimate. RL spends a lot of time sampling, so its FLOPs share is smaller than its GPU-hours share.
Notice a trend: the open-source world is starting to stratify. Nathan calls “building core reasoning behavior” post-training, and “domain adaptation on an already post-trained model” post-post-training. Over the past 12 months, most open-weight labs have released only post-trained weights and no longer release base models. Harvey used Fireworks to post-train Kimi K3 into a legal model, so law firms can own their own model; Mercor and SkyRL did RL on Qwen3.5-397B-A17B with expert data, with scripts and weights both public. This is the prototype of post-post-training.
Next are the six open problems he cares most about, ordered by compute intensity from low to high:
- Harness specialization and design: how do we design context-management environments so that LLMs can be more effectively orchestrated into specific tasks? Context Language Models is an example: the model edits context itself, rather than relying on harness commands.
- Character training: where do a model’s persona and values come from, and how do they affect safety, trust, and adoption? Industry has already shifted to OpenAI’s agent Model Spec, while the open literature has not caught up. Trillium treats this as one of its first research directions.

- Open-source RL environments: post-training does not have public data resources like pretraining does. Nemotron, Olmo, and OpenThoughts are starting points, but they are not enough. TMax uses nine axes to compose tasks, generates terminal-bench data, and turns Qwen models from 2B to 27B into terminal agents.

- Reasoning efficiency in RL: how do we control CoT length? Frontier labs have already achieved huge token-efficiency gains, but the public papers do not explain the methods clearly. One simple idea: spend more RL compute on the hardest problems. GRPO samples multiple answers for the same prompt; when the whole group is wrong, there is no gradient to learn from. Never Give Up (NGU) keeps sampling more in this situation instead of giving up. On Manufactoria, GRPO stalls at 19% on hard tests, while NGU reaches 88%.

- Scaling laws for RL: how do we predict the results of large post-training runs? ScaleRL fits reward against log compute with a sigmoid, but the RL equivalent of a pretraining ladder has not yet appeared.
- Multi-agent scaling laws: multi-agent systems can be faster in wall-clock time and can also raise peak performance. OpenAI's GPT-5.6 on SEC-Bench Pro reached the same score earlier with 16 agents than with 1 agent; Anthropic's Claude Opus 5.5 on Lean theorem proving basically plateaus above 10 agents. How to design RL for sub-agents during training remains a blank space.
The talk ends with RSI. In Anthropic's R&D Automation Index, model R&D tasks led by Claude rose from less than 1% in February to 26% in August, while fully automated AL5 is still 0. Nathan's view is: LLMs accelerate output, but hardly accelerate understanding. Senior scientists barely write code and are instead busier; junior scientists build understanding through manual, inefficient work, so training time may get longer.

In the comments, Lewis Tunstall shared Ramez Naam's long essay from that day, “The Most Important Charts in AI are Wrong”. Ramez compares OpenAI and Google internal data with AI 2027 forecasts from the evaluation organizations METR and AI Futures: at 80% success rate, the task length of real R&D tasks is about 16 minutes for OpenAI and about 9 minutes for Google, while the ECI→METR extrapolation is 33× and 41×, and the AI 2027 scenario exceeds 100×; the gap is still widening. Benjamin Todd pushed back in the comments, arguing that at 50% success rate, a doubling speed of 4 months is already fast enough, and Anthropic's lead may be even larger.
This debate has no conclusion. But at least it shows one thing: looking only at benchmark curves versus looking at messy real-world R&D tasks yields very different timelines. The reliability gap matters more than the capability curve.
There are two more things in the comments worth following up on. Chinmay’s experiment uses an agentic judge to give fine-grained scores to reasoning spans generated by models; it found that the spans the judge marks are indeed moving the model closer to the correct answer, but the main change after training is that the model locks onto the answer faster, saving about 1,900 tokens. Cornell’s PhantomEnvironments trains search agents in fictional worlds generated purely by rules, and transfer to real multi-hop search even outperforms real training data. These two directions—one changing the reward, the other changing the environment—land right on the open problems Nathan lists.
Nathan says the video will be re-recorded. The slides are at natolambert.com/cornell-2026. If you are also working on these directions, he says to stay in touch.
发布时间: 2026-11-22 16:07