SGLang Pushes DeepSeek V4.1 Flash to 1.09M tok/s, and the Comments Are Asking About That 40%

The SGLang project posted a set of numbers on X: over the past four days, DeepSeek V4.1 Flash running SGLang on AgentX saw single-GPU peak aggregate throughput rise from 654K tok/s to 1.09M tok/s. The measurement basis includes cache, new input, and output. There was no warning and no long explanation, just one line: 'there is more.'

In the same test set, the ultra-low-latency scenario also got numbers: 943 tok/s/user at concurrency 1, thanks to MegaKernel. For people doing inference deployments, these two numbers point in different directions: one looks at peak throughput, the other at single-user interaction speed. Real systems rarely max out both ends at the same time.

The chart comes from SemiAnalysis InferenceX and is titled DeepSeek V4.1 Flash 552B Agentic Token Throughput per Chip vs. P90 Interactivity. The x-axis is P90 Interactivity in tok/s/user, from 0 to 1000; the y-axis is Token Throughput per chip in tok/s/chip. The green line marks configurations including 9xDEP2+1xDEP16, 8xDEP2+1xDEP16, 5xDEP2+1xDEP16, 3xDEP2+1xDEP16, TEP2, and TP4. The chart was updated on 2026-10-30T00:00:00.000Z. It puts throughput and interactivity on one chart so readers can directly see the trade-offs of the line under different configurations. Note that the main post gives per-GPU peak aggregate throughput, while the chart gives per-chip throughput versus P90 interactivity; don't conflate the two measurement bases.

The questions in the comments are more informative than the main post. Some users asked what share of the 1.09M was cache hits, some asked about performance at higher concurrency, and someone did the math: a 65% improvement, but usage only increased 25%; where did the remaining 40% go? Another reply mentioned that the 943 tok/s/user number is more critical, and that the remaining latency mainly comes from host-side DSpark checks and the small fused all-reduce operations that still bubble up on larger drafts. This claim was not expanded on in the main post, and no link has been provided for MegaKernel details.

So what can be confirmed now is: SGLang pushed DeepSeek V4.1 Flash's peak aggregate throughput to 1.09M tok/s/GPU on AgentX, and gives 943 tok/s/user at concurrency 1. What cannot be directly confirmed: cache hit share, the scaling curve across concurrency, the end-to-end latency composition, and what MegaKernel specifically does. For production deployments, these latter items are harder to fudge than the peak. Peak is the billboard; cache hits and concurrency curves are the bill.

Original post: https://x.com/sgl_project/status/2116353629057962217