108k tokens/s: Gemini 4 Argon gives Gemma a 2.85x speedup, but token parity still hasn't won over the crowd

On October 26, 2026, the official Google Gemma account posted a result: Gemini 4 Argon helped gemma-4-E2B-it go through a round of performance optimization in vLLM. On a single A100, throughput rose from about 37,900 tokens/s to 108,000+ tokens/s, 2.85x. The official post also listed three conclusions: 100% token parity, zero quality degradation, and the changes took only a few days rather than months. All modifications are public in the vLLM repository, 8 PRs, authored by collaborator lucianommartins. View the 8 PRs

The optimization approach in this batch of PRs is straightforward: fuse scattered operators into Triton kernels, including QKV norm/RoPE, pre/post-FFN norms, and MoE routing; tune configs for MoE models; add KV-sharing fast prefill and merge MTP speculative decoding layers; multimodal inputs were not left out either, batching the vision pooler and fusing Gemma4Unified 2D posemb. Along the way, multimodal pooling support for EmbeddingGemma2 was also merged in.

The tweet included a video showing the test workload's running UI: batch size 64, input 2048 tokens, output 16 tokens, sustained throughput over 108k tokens/s on a single A100. The video annotation shows that TTF (time to first token) is 3.17x faster than baseline, and E2E throughput is 2.92x faster.

In the comments, the most rigorous people were all asking about verification methods.

Developer Michał Piszczek asked a specific question: in the PR for the fused QKV kernel, the output is bit-exact at batch size 1, but at batch size 512 only 44–46% of tokens are fully exact, with about 80% falling within 1 ULP error. So was the official 100% token parity measured at batch 1 or under the 108k tokens/s workload? Daniel Priscu added: only long context can test parity; short prompts hide the problem. Aleksandar Janca was more direct: token parity is the lifeline for agent-written kernel optimizations; if the output comparison step isn't solid, the faster it gets, the more dangerous it is.

bemuuvi cared about whether smaller GPUs could get the same speedup. martin openly mocked: why run a 2B model on an A100? No matter how fast, it's still a small model. Bowen thinking asked another thing: will these optimizations be synced to Gemma 26B-A4B? When will they be available on Google Cloud? He mentioned that the official release of Gemma 4 has been out for 208 days. Attached was a rocket launch animation.

This round of optimization itself is a good sample: an agent completed kernel-level optimization within a few days, all PRs are public, and the community can inspect them line by line. But the skepticism in the comments also shows one thing: benchmark numbers are getting cheaper and cheaper, while verification methods are the scarce commodity. Saying 100% token parity is easy; proving it is hard.