While others write flag docs, they tested every decision on L4 and DGX Spark

Rohit Ghumare posted a tweet about Miles 101 on X and brought vLLM 101 along with it. Both handbooks are on aiengineeringfromscratch.com, MIT-licensed, and free to read.

Their positioning is different from the official docs. vLLM's documentation tells you what each flag does; these two handbooks tell you which flags really matter for your GPU and your traffic, and what happened after changing them. The test hardware was an NVIDIA L4 and a DGX Spark GB10, and every number is labeled with the machine.

A few numbers worth looking at in vLLM 101

The most counterintuitive one is in the memory chapter. On the L4, the KV pool for Qwen3-8B is 24,096 tokens. The startup log prints Maximum concurrency 2.94x, which is calculated assuming every user fills the 8,192 context. Actual requests are 512-token input plus 128-token output, so each user occupies 640 tokens, and the same pool can hold 37 users. That number on the startup line is scary, but it is not your concurrency limit.

Another one is in the chat tuning chapter. On the L4, running 16 concurrent users gives a P99 ITL of 85ms, and one third of requests already have a first token over 1 second. At 32 users, P99 ITL jumps to 613ms, and no request meets the target. There is an easily overlooked detail here: when they reran it later, they found that without resetting the prefix cache, the numbers below 16 users were skewed by cache hits. After resetting, the latency knee moved earlier, from 32 users to somewhere between 8 and 16 users.

On DGX Spark, the total output at 64 users is 8x that of a single user, but each user's time waiting for each token also rises by about 7.5x. Cheap and fast run in opposite directions on the same GPU.

Miles 101 solves a very specific problem

Miles is an open-source RL framework for running GRPO on a single GPU. The most practical item in the handbook: when the rollout engine dies, Miles' trainer does not stop; it keeps sending requests to the dead engine. In their test, an engine crashed at startup, and the job kept running for 63 minutes, wrote 200,533 HTTP 503s, left the GPU idle the whole time, and reported no failure.

The fix is a watchdog script that counts the number of 503s in the last 60 seconds every 30 seconds, and stops the container after 10 consecutive counts above 100. A healthy 503 burst generally lasts no more than 60 seconds, with at least 45 seconds between bursts; in the replay, 10 consecutive over-limit counts produced no false kills.

Another conclusion is about model size. On the GB10 with 128GB of unified memory, Qwen3-1.7B can run 4 steps with short 256-token answers, while Qwen3-4B cannot even get past Ray's memory line with short answers. The handbook's estimate is that full fine-tuning uses about 16 bytes per parameter: 1.7B is roughly 27GB, 4B is roughly 64GB, and after adding the engine and activations, a 128GB machine is not enough for 4B.

Why it’s worth reading

The value of this kind of handbook is not in “teaching you how to install vLLM”; the installation commands are in the official docs too. Its value is in laying out the pitfalls of defaults on different hardware: the same --gpu-memory-utilization is a hard cap on cards with dedicated VRAM, but on DGX Spark's unified memory it is only a plan, and actual usage may exceed it. The same FP8 KV cache silently switches the attention backend on the L4, from FlashAttention to FlashInfer, and FlashInfer's first compilation may crash outright.

Someone commented under the tweet that running 16 concurrency safely on a 5090 is no problem. That number lines up with the knee on the L4, but that is a different card and a different memory bandwidth. Every page of the handbook reminds you of the same thing: use other people's numbers as reference, and use your own numbers to decide.

Dipankar Das asked a question in the replies that has nothing to do with the content: now that everyone uses AdamW, does anyone still mention Adam? That question fits well under an RL handbook, because in the optimizer step of GRPO, people really do often fail to look closely at the defaults.