Wink Pings

Pre-training Is Only the Foundation; Whether an Agent Can Do the Work Comes Down to These Five Steps

Hugging Face has released a free post-training agents course: a blog paired with a YouTube livestream series, covering the path from SFT, distillation, and RL all the way to RL environments and training at scale. Each step includes a flowchart, a lesson, and documentation, and some also come with runnable projects.

There is a free course on Hugging Face about post-training agents—a blog plus a YouTube playlist—by ben burtenshaw, Sergio Paniego, merve, and others.

A bit of background first. Post-training refers to the training stages after pre-training; supervised fine-tuning, distillation, and reinforcement learning all count. What it does is turn a model that can only continue text into one that follows instructions, uses tools, and can complete concrete tasks. The trouble with agents is that they need to act over multiple turns: call a tool, read the result, then decide the next step. Training an agent with the single-turn Q&A playbook only teaches it how to say the first sentence.

Post-training gets discussed less than pre-training, but what really determines whether a model is useful is exactly how the sequence of SFT, distillation, and RL is arranged and validated.

This material is organized around multiple turns. Five steps, each with a flowchart, a livestream or a set of slides, an explanatory article, and corresponding documentation; some chapters also include runnable projects. The training examples consistently use a coding agent.

  1. Before You Start

Align the terminology first. The terms agent, harness, and scaffold are often used interchangeably; Hugging Face wrote a dedicated glossary to clarify them, plus a 20-page overview slide deck. If you haven't worked with fine-tuning, start with the LLM Course.

  1. SFT

The model learns from agent traces: a complete recording of an agent solving a task, including every tool call and returned result, usually produced by a stronger model. This step is cheap and stable. The cost is that the model only sees correct trajectories and never sees what its own mistakes look like, so it cannot learn how to climb out of errors. On-policy distillation and RL fill this gap.

SFT flowchart: the model reads agent traces from a stronger model and computes loss for the next token

  1. Distillation

A teacher model supervises the student. Either the teacher writes data, or it gives probabilities on tokens generated by the student itself. The teacher can be a larger model, a combination of several expert models—the MOPD approach—or the same model with extra context, i.e., self-distillation. The signal density is one value per token, which uses fewer samples than RL. Conversely, the student's ceiling is capped by the teacher's knowledge, and on-policy distillation still has to run the teacher at every step.

Distillation flowchart: the student generates tokens, the teacher gives probabilities and corrects low-probability tokens

  1. RL

Drop the teacher and instead use a reward function to score the model's own outputs. Write an answer, score it, and the probability of high-scoring answers is raised. The algorithm used here is GRPO: sample several answers per prompt, score them all, and increase the probability of those above the group average. Comparing within the group is enough to judge which attempt is better, without training another model to estimate it.

The signal is weaker than distillation: the entire answer gets only one number, so the model needs many attempts. When the reward function is simple, each answer is one round and is scored only once.

GRPO flowchart: sample multiple answers for the same prompt, score them by test pass rate, compare with the group mean, and backpropagate

  1. RL Environments

The reward comes from acting in a stateful environment. The environment runs tools, records changes, and checks the final result—for example, running tests for a coding task. Each episode is longer than a single answer, and generating rollouts becomes the slowest part of training.

Environment flowchart: the model acts in a multi-turn environment, and the environment executes tools and checks the final result

  1. Harness and Scale

Each rollout is a complete agent run, executed in an existing agent harness with a remote sandbox wrapped around it. By training with the same harness used in actual deployment, the model learns the very environment it will run in later. Generation and training are split across two groups of GPUs, so long episodes don't leave the trainer waiting idle; updated weights are then pushed back to vLLM.

Scaling diagram: generation and training are separated across two groups of GPUs, and weights are synchronized back to vLLM

The full playlist is here, and the original blog post is here.

Some readers mentioned that when taking this course, it's best to directly take one of your own small tasks and compare before and after; otherwise, after listening, you still won't know which step made the difference. That's practical advice. The five steps are laid out there, but to really know which piece you're missing, you have to run it yourself.

发布时间: 2026-10-30 00:37