Kimi K3.1 Pro Day 0: SGLang Squeezes a 2.8T Model onto a Single Hardware Checklist
Kimi K3.1 Pro's weights are open sourced, and SGLang released a deployment recipe and a hardware checklist spanning NVIDIA, AMD, TPU, Trainium, and PPU on the same day. The 33.7K tok/s per GPU and 450 tok/s single-user figures each come with workload prerequisites; the cookbook also clearly states which cells have been validated and which are still being verified.
On November 4, Moonshot AI released the weights for Kimi K3.1 Pro, and SGLang announced Day 0 support the same day. The two numbers in SGLang's tweet are striking: running the AgentX workload on GB300 delivers up to 33.7K tok/s per GPU; single-user throughput is 450 tok/s, with DFlash2 plus MegaKernel as the condition.
Kimi K3.1 Pro is a 2.8T-parameter MoE with 104B active parameters and a 1M context. Architecturally, 69 of its 93 layers are KDA, and 24 are Gated Sparse-MLA, i.e., DSA. There are 896 experts, with 16 selected per token plus 2 shared experts. The weights use MXFP4 and activations use MXFP8. SGLang's tweet says it replaces dense MLA with sparse attention, making it more efficient and easier to serve. For general readers, think of it this way: the model is huge, but only a small subset of parameters wakes up per token; it can handle long context, but VRAM and cache are still hard limits.
SGLang's Day 0 hardware list is long: NVIDIA Vera Rubin, B300, GB300, B200, GB200, H200, H100; AMD MI355X, MI350X; Google TPU7x; AWS Trainium 2/3; Zhenwu PPU M890, 810E. A reply to the tweet specifically notes that this is the first time Day 0 support includes Trainium, with code coming soon. A long list does not mean every cell produces the same numbers. The cookbook states clearly that B300 Decode Balanced with DFLASH is marked "Verified"; B300 Decode High-Throughput without speculative decoding is also marked "Verified"; and the MI350X/MI355X DFLASH recipe is marked "Verified". Other cells are still being verified. The cookbook also notes that there are currently no cookbook-format speed sweeps, and the benchmark card shows pending.
On the deployment side, the SGLang cookbook provides a command generator: choose hardware, deployment mode, and serving strategy to generate a launch command. Strategies include Low-Latency TP8, High-Concurrency DCP8, High-Throughput, and Prefill/Decode disaggregation. There are also options such as the Mamba ratio calculator, HiCache, DFLASH draft, and WideEP Decode. Developers can copy the commands directly, and general readers can also see that Day 0 support is not just a claim that it runs, but a set of recipes and an acceptance table.
The ecosystem responded quickly. Databricks' Yuchen Jin said in a reply that they launched Kimi K3.1 Pro on Day 0, the team is already using it for internal coding, and he called it the best open-source model available right now. That is his judgment. On the other side of the comments, some asked whether it could be quantized to run on a Mac Mini, some asked how much the accepted length of DFlash2 differs between code and prose, and others asked whether the indexer scan at each step under a 1M context would become the main cost. These questions are closer to real deployment than "it's open source." For a 2.8T-parameter model, open weights do not mean it can run on a laptop; the quantization community will show up, but people deploying it first look at KV cache, PD disaggregation, and verification status.
The weights are on HuggingFace, the deployment recipe is in the SGLang cookbook, and SGLang's Day 0 blog post also provides technical details. Competition among open-source models is no longer just on leaderboards; it is also at the Day 0 serving layer. Whoever can make a new model produce reproducible numbers on more hardware gets the ticket to the next round of deployment.
发布时间: 2026-11-05 17:34