When training and inference don’t line up—that’s the most annoying part of post-training

Together AI has launched its Post-Training page. The core message is one sentence: training and generation stay aligned, so the reward signal you design is what the model actually learns, and that behavior carries over into serving. The slogan isn’t new, but turning that sentence into a product shows that the competitive battleground for post-training has changed.

Many teams can get an RL loop running. After launch, model behavior drifts: the reward appears to go up, but the outputs on the serving side differ from those during training. The causes are usually in the details: numerical precision between the rollout engine and the trainer, MoE expert routing, stale samples, and weight sync. The official page lists these under Quality: the trainer matches the generator’s output precision, replays expert selection on supported MoE models, and each RL loss corrects for stale samples; bad steps are stopped by a safeguard. This is dirty work.

The division of labor is clear. You control the recipe: data, prompts, environment, reward, verifier, judge, GRPO or PPO or a custom loss, multi-LoRA or full weights, and where the model is deployed. Together handles on-demand or dedicated GPUs, training–rollout consistency, parallelism configuration, the rollout engine for each model, and synchronous or asynchronous training.

An RL step in Python is sample(), your_reward(), forward_backward(), optim_step(), weights_sync(). SFT uses the same loop, minus sample() and weights_sync(). The official team provides a Tinker-compatible cookbook, linked here.

For parallel experiments, one call reserves the GPUs the model needs, and the setup has already been tested per model. On the same reservation, you can open a LoRA session for each experiment at no extra hourly charge, and results go to your own W&B. After training, save the checkpoint, attach the adapters to a multi-LoRA endpoint for side-by-side comparison, deploy the winner to a dedicated endpoint, or download the weights and run it yourself.

On performance, the official page mentions precompiled GPU execution, prefix caching, packed training, GPU-to-GPU weight sync, and async so that generation and training overlap. Billing is by GPU-minute, charged only when the model resource is ready; queueing, startup, and shutdown are free, and an idle resource stops after one hour. Sharing one reservation across multi-LoRA experiments and having SFT skip the generator both shift more of the spend toward training.

The supported model list includes GLM-5.3, Kimi K3, Qwen3.8 27B, and Qwen3.6 35B A3B. The FAQ says reward and environment are your own code and can be paired with any sandbox. For multi-turn agents with RL, Together generates each turn, tool calls happen in your sandbox, history turns are cached on the GPU, and new turns only process newly added tokens. LoRA saves GPUs; use full-weight when adapters aren’t enough. Start with synchronous RL, and switch to async when step time matters; the loss corrects for samples that lag one step behind.

Under the source tweet, someone followed up: how do you maintain token parity under FP4? The official diagram writes the tokens in the TRAINING, GENERATION, and SERVING·FP4 columns as identical, circles IDENTICAL TOKENS in the top right, but the page doesn’t explain the mechanism. This follow-up question is more worth reading than the launch itself. Post-training tools all say 'you just write the reward,' but the real difficulty is that invisible drift between training and serving. FP4 serving magnifies the problem. Whoever can explain token parity clearly will have a more credible product.