Wink Pings

Break LLM inference into 12 steps, and you can see tokens queue up and KV cache grow

An LLM serving simulator you can pause, turning the four concepts of prefill, chunked prefill, continuous batching, and KV cache into frames you can advance one step at a time.

LLM inference articles usually give you two static diagrams: one for prefill, one for decode, with an arrow in between. After reading, you still don't know how multiple requests on the same machine compete for resources.

Mak built an interactive simulator that breaks this down into 12 steps you can execute one at a time. You can watch requests queue, tokens stream out, and KV cache grow one cell at a time.

1/ Prefill and decode

The model first processes the input prompt, then generates one token at a time. The simulator lets multiple requests share the same server, so you can see at the same time who is reading the prompt and who is generating output.

2/ Full prefill and chunked prefill

When a long prompt comes in, processing it in one block ties up compute, and requests that are generating get stuck. After splitting it into small chunks, each round processes only one segment, and other requests can keep decoding.

A commenter replied that most inference articles gloss over the difference between chunked prefill and a long prompt holding up the whole batch. That observation is quite accurate.

3/ Static batching and continuous batching

Requests finish at different times. If you wait for the whole batch to finish before admitting new requests, slots sit idle. With continuous batching, a new request is inserted as soon as a slot frees up. In the simulator you can directly see under which strategy capacity sits idle and under which strategy it gets reused.

4/ KV cache

The cache keeps accumulating as requests progress. The simulator displays it side by side with token generation, so the relationship between memory growth and output speed becomes a visible sequence of steps.

How it was built

Mak built it with Base44, starting from a written spec and visual reference, adjusting the interface with follow-up prompts, and using the built-in database to save experiment state. Reopen an experiment and both the settings and playback position are still there.

base44

This save feature is useful for learning. You can save a configuration and come back to the same scenario a few days later without re-tuning parameters.

The simulator is a simplified learning tool, not a production-grade load-testing platform. But it connects terminology to visuals. If you are learning inference or need to explain these concepts to a team, pausing, changing a setting, and watching the change is faster than reading ten pages of docs.

Demo URL: https://serving-trace-lab.base44.app/

发布时间: 2026-11-19 20:29