30 Minutes a Day, 10 Weeks, 50 Lessons, and Only One Inference Service at the End

Akshay Pachaar has posted an LLM inference roadmap on X. 10 weeks, 50 lessons, 30 minutes a day, and in the end you deliver just one thing: an OpenAI-compatible inference service. Deploy it on rented GPUs, add monitoring, load-test it to more than 1,000 concurrent requests, implement quantization, speculative decoding, KV cache eviction, then put a router in front that chooses backends based on cost, latency, and quality, and finally publish benchmarks that can be reproduced.

What makes this plan worth reading is the order, not the length of the list. Week 1 starts with roofline and arithmetic intensity, making clear why decode is limited by memory bandwidth and why prefill is limited by compute. Every later optimization lands on the same chart. Quantization targets memory-bound decode, continuous batching pushes arithmetic intensity toward the compute roof, speculative decoding trades cheap FLOPs for sequential memory loads, and prefill/decode separation comes from the two phases competing for the same GPU. Without this mental model, what remains is just a pile of tricks.

Week 2 deploys vLLM, then reads the block manager and scheduler code. The roadmap explicitly says not to rewrite PagedAttention and continuous batching—reading the code is more useful. The reason is straightforward: vLLM has already implemented both, and chunked prefill is still the default scheduling policy in vLLM and SGLang. Rather than rewriting them, read until you can explain why KV memory waste can be reduced below 4% and why the GPU does not idle between requests.

Observability comes only in Week 3. Some commenters think observability should come first, and that disagreement is itself interesting. The roadmap places it before optimization: Week 3 connects Prometheus and Grafana, and only Week 5 runs the 1,000-concurrency load test. Without dashboards for TTFT, inter-token latency, throughput, and queue depth, later tuning cannot be validated. The core of a public benchmark is often a trustworthy measurement setup; the model is just hung on top of it.

The Week 5 load-testing requirements are very specific: script a sweep across concurrency levels, report p50, p95, and p99, and cross-validate with two independent tools. One commenter said p50/p95/p99 should be tattooed on every benchmark post—averages hide the user who waited a long time. Another easy-to-miss point is that the workload must be fixed; otherwise you are tuning noise.

Week 4 compares SGLang with vLLM and covers RadixAttention and prefix reuse. One commenter added a practical detail: prefix caching often silently fails to take effect. If you put a timestamp, session ID, or username at the top of the prompt, every request becomes a cache miss, and the symptom is prefill time growing with the full prompt length. Put stable bytes first and variable content later.

Weeks 6 through 8 cover quantization, speculative decoding, KV eviction, prefill/decode separation, and Kubernetes. There are two counterintuitive conclusions worth remembering. Speculative decoding can reach 2.8× at low QPS, but at high QPS it becomes 1.4× to 1.8× slower; the crossover point is more useful than the speedup ratio. INT4 is better used as a memory-pressure lever; when quality proxy metrics drop by more than 1% to 2%, the default should still be FP16 or FP8.

Week 9 combines unit economics and the router. Divide the GPU hourly price by peak tokens/s, utilization, and 3600, then multiply by 1 million to calculate cost per million tokens. The router chooses backends by cost, latency, and quality; per-request token budgets go into code, and cost goes onto the dashboard. Week 10 publishes the benchmark with pinned versions and commands so others can reproduce it. The roadmap also warns that the performance numbers cited in the article are numbers the sources are willing to defend themselves, not independent reproductions; GPU prices and cost per million tokens change quickly, so recalculate with your own hardware.

The toolchain is not complicated: vLLM, SGLang, GuideLLM, genai-perf, Prometheus, Grafana, Kubernetes, plus kind or a managed cluster. GPU cost is roughly renting a 24GB card by the half hour, plus two H100 sessions. Most of the reading and most of Weeks 1 and 3 do not need a GPU.

The repo is on GitHub under CC BY 4.0; additions of links, new sources, and benchmarks are welcome. Repo: https://github.com/patchy631/time-to-first-token . Original tweet: https://x.com/akshay_pachaar/status/2118699262259466406 .

Model release news is never-ending. This roadmap at least converges 50 learning sessions into one service that can be deployed, load-tested, and publicly reproduced. Bookmarking 17 disconnected experiments usually leaves you with nothing but environment setup. A service that keeps growing at least leaves behind dashboards, load-test scripts, and a benchmark others can poke holes in.