Inference Engines Don’t Get Weekends Off: SGLang and vLLM Go Head-to-Head, Meta Posts a $100K Leaderboard

This weekend, inference engine teams did not rest. Han Xiao wrote in a tweet that SGLang and vLLM are racing, and inference people don’t get weekends off. The post included two sets of throughput comparisons.

The first chart compares vLLM and DeepInfer on DeepSeek-V4.1-Flash. On the left is decode, output tokens per second per GPU; vLLM is the yellow dashed line, DeepInfer the blue solid line. The trends are close, with vLLM slightly higher for some batch sizes. On the right is prefill, new-token generation speed as a function of latency step size, where vLLM also has the edge. The caption adds a key caveat: the data was read off charts released by DeepSeek, and the hardware and parallelization differ, so this is not an engine-to-engine speed comparison. Reading that sentence before looking at the leaderboard would prevent half the arguments.

The second chart uses a different configuration. It compares SGLang and DeepInfer on 4x GB300 and 64x GB300 for prefill and decode, with latency on the x-axis and throughput or per-user speed on the y-axis. The chart marks SGLang’s percentage improvements under some configurations. I won’t go into the exact numbers; the trend is clear: there is still room to optimize inference engines, and it’s not just minor tweaking.

Even livelier than the charts is another leaderboard. Meta, together with Paradigm and Andromeda, is running the Muse Spark Open Kernel Challenge, with a $100,000 prize pool. The task has two stages: first optimize GPU kernels for Muse Spark Open, then accelerate SGLang serving. The Kernel track runs from October 22 to November 4, and the Serving track from November 5 to November 19. There are five kernel problems: muse_bf16_proj, muse_attention_decode, muse_moe_w4a16, muse_decode_layer, and muse_prefill_layer.

The rules are very specific. Kernel submissions run on a single H200; correctness is checked against a reference implementation using random inputs; timing uses CUDA graphs with a cold L2, and speedups are calculated for workloads such as 1, 16, and 32. The Serving track is heavier: it uses a full 8×H200 node to run hidden multi-turn agentic coding conversations at concurrency levels of 1, 4, 16, and 32, with scoring based on per-GPU throughput and per-user interactivity. Prizes are $50,000 plus a signed H200 for first place, $30,000 for second, $10,000 for third, $3,000 for fourth, $2,000 for fifth, and $1,000 each for sixth through tenth. Top competitors may also get Meta interview opportunities.

People have already posted results on the leaderboard. On muse_moe_w4a16, alpinyukselogu ranks first with 8,567.2 µs, 1.612×; arjunblj at 8,927.2 µs, 1.547×; frankieislost at 9,250.1 µs, 1.493×; matthuang at 9,650.9 µs, 1.431×; muse-spark itself at 9,978.6 µs, 1.384×; and SGLang reference at 13.81 ms, 1.000×. In a reply, Robert Lukoszko said that on the first day of the competition he beat SGLang on every benchmark. Whether that holds for all benchmarks depends on the full leaderboard, but someone blowing past the baseline on day one is a direct enough signal.

Someone in the replies posted a cartoon video. A brown puppy wearing a yellow hat sits at a table with a white cup on it, while flames and dark clouds fill the background. There’s no caption, but the mood is clear. This is probably what an inference engineer’s weekend looks like: the cup is still there, and the fire is already at their back.

There are also cooler-headed jabs. Raghav Mehta said DeepSeek dropped an engine on Tuesday, and by Saturday both teams had caught up. niels .1 said the two engines feed on the same set of open kernels and then call it a race. qrdl suggested paying $1,000 to whoever tops the 24-hour leaderboard; otherwise everyone will wait until the last minute to submit. Moriarty asked about hardware limits, and SemiAnalysis suggested adding tpuv7 and sglang-torchtpu too. Ash Hart asked whether they could compete without SGLang, bringing TensorFold. According to the rules, the Serving track requires submitting SGLang’s patch.diff; the Kernel track allows Triton, CUTLASS, FlashInfer, DeepGEMM, or your own CUDA. Anyone who wants to switch engines needs to read the track rules first.

Infrastructure is under pressure too. Abhijith Neil Abraham said H100s and H200s have been fully booked this week by launches and openkernelchallenge, and providers often have no GPUs to spin up. Marshall Mathur added that ordinary people are still learning and experimenting on T4s, with a screenshot of everyday life. That contrast is more real than the leaderboard: on one side, 8×H200 nodes are queued up; on the other, people are running PRs and docs on T4s.

Jonathan M mentioned BackboardIO as a dark horse and posted two images. There’s no more context, so I’ll leave them here for now. That’s the fun part of open-source challenges: a sponsor sets up the mechanism, the baseline is public, the target is public, and the next person keeps improving it. In a reply, Ash Hart said this is the benefit of open source: sponsors, fines, and engineering mechanisms, and then the next person does better, and the cycle continues. It sounds like a platitude, but next to this leaderboard, it fits quite well.

The weekend isn’t over yet. The SGLang vs. vLLM comparison charts will keep circulating, and the Muse Spark leaderboard will keep changing. Competition among inference engines has no holidays, only the next submission.