Agent post-training in 5 steps: SFT is the most stable, but also the easiest to make you misjudge

Hugging Face's Sergio Paniego breaks Agent post-training into 5 steps, from SFT to RL, and then to real harness and scale. Each step comes with a flowchart, livestream, code, and docs. The original article is here, and it is worth saving first and reading slowly later.

First, some background. Post-training refers to the training stage after pretraining, including supervised fine-tuning, distillation, and reinforcement learning. It turns a pretrained model into a model that can follow instructions, use tools, and solve specific tasks. What is special about Agents is multi-turn interaction: the model has to call tools, read results, and decide the next step. Agentic post-training must train the entire interaction, not just focus on a single response.

This curriculum unfolds in 5 steps. The order comes from Hugging Face YouTube's Training Agents livestream series, and the example task is training a coding agent. The glossary is in Harness, Scaffold, and the AI Agent Terms Worth Getting Right, and the 20-page overview is in Train the agent, not just the model.

1. SFT: first learn to copy trajectories

SFT lets the model copy agent traces. A trace is a record of the process by which an agent solves a task, including every tool call and result, usually generated by a stronger model. It is cheap and stable. The problem is that the model only sees correct trajectories and does not see its own failed attempts. If it makes a mistake while solving a problem, it has also never learned how to recover. On-policy distillation and RL train on the model's own outputs, filling this gap.

Livestream: Training Agents 1: fine-tune a coding agent, code in gemma4-pi-mono-youtube-livestream-1-scripts. Docs: SFT Trainer.

2. Distillation: the teacher gives a signal token by token

In distillation, a teacher model supervises the student. The teacher can teach by writing data, or by giving probabilities for tokens generated by the student itself. The teacher is usually a large model, or multiple expert models distilled into one student, or the same model with extra context for self-distillation. The signal is dense, one value per token, so it needs fewer samples than RL. The cost is that the student is limited by the teacher's knowledge, and on-policy distillation requires running the teacher at every training step. RL replaces the teacher with a reward and only checks the result.

Livestream: Training Agents 2: distillation. Further reading: A brief history of distillation in AI and Distillation in 2026 (so far). Docs: Distillation Trainer.

3. RL: use scores to push the model to try more

RL lets the model learn from scores on its own outputs. The model writes an answer, a reward function scores it, and training makes high-scoring answers more likely. This does not need a teacher, only a way to check answers. The course uses GRPO, which stands for Group Relative Policy Optimization. Multiple answers are sampled for each prompt, all are scored, and answers above the group average get a higher probability. Comparing within the group tells the model which attempts are better, without separately training a model to estimate a baseline.

RL signals are weaker than distillation: one score per answer, not a value for every token. The model needs many attempts. Under an ordinary reward function, each answer is one episode and is scored only once. An environment lets the model act multiple times and see the result of each action.

Livestream: Training Agents 3: reinforcement learning. Reading: How frontier models train on outcomes in 2026. Project: Fine-tuning a 350M model for structured outputs in 100 GRPO steps. Docs: GRPO Trainer.

4. RL environment: reward comes from multi-turn actions

An environment has state, exposes tools, and can respond over multiple turns. The environment runs tools, records state changes, and checks the final result, such as running tests for a coding task. Each episode is longer than a single answer, and generating rollouts becomes the slowest part of training.

Livestream: Training Agents 4: from reward functions to environments and RL for Agents Workshop. Reading: Why RL environments work now (and could not in 2016). Projects: Paint watercolours with code and Autonomous driving with CARLA. Docs: TRL + OpenEnv and OpenEnv docs.

5. Harness and scale: train in the real runtime environment

Every rollout is a complete agent run, executed in a remote sandbox through an existing agent harness. Training uses the same harness that the agent actually uses, so the model learns in the environment where it will run in the future. Generation and training are placed on different GPUs, so long episodes do not leave the trainer waiting idle. The trainer then sends the updated weights back to vLLM.

Slides: Training a coding agent through a harness you did not write. Reading: The ultimate guide to multi-harness RL, One sandbox per rollout, Keep the Tokens Flowing: lessons from 16 open-source RL libraries. Docs: Async GRPO Trainer and OpenEnv harnesses.

One reply worth watching

Under the original tweet, a user named unify mentioned an experimental observation: a model trained with SFT only on OpenCode dropped by about 3 points under Mini-SWE-Agent; with multi-harness SFT, and with Mini-SWE-Agent's own trajectories also included in the training data, it dropped by 17 points in that environment. This goes against intuition, showing that the relationship between harness matching, data distribution, and generalization is not so straightforward. It is an empirical observation in the replies, not a conclusion of the course, but it is enough to remind people: do not treat SFT scores in a single environment as general capability.

The course entry point is A curriculum for post-training agents, and the livestream collection is Post-training Agents playlist. If you want to run through it from the beginning, first look at the glossary, then go in the order of SFT, distillation, RL, environments, and harness. RL is not the starting point; the real harness is where problems are most easily exposed.