Wink Pings

TensorFold 1.3.3: Decode speed up 13.8%, but prefill time is still a black box

TensorFold 1.3.3 is released, with 124 PRs merged in 24 hours and GLM-5.3-Flash decode speed improved by 13.8%. X user @DonaldJ.Gorbachev, however, did the hardware math and asked: how long did the prefill for 1,040,384 tokens actually take?

TensorFold 1.3.3 has been released. Ash Hart posted the scorecard on X: 124 PRs merged in 24 hours and 34 issues resolved. GLM-5.3-Flash, running on four DGX Sparks (GB10) with TP=4, increased its median drafted decode from 151.51 tok/s to 172.39 tok/s, a 13.8% improvement. MiaAI_lab provided the kernels behind it.

This update also includes a string of features: GLM-5.3-Flash supports DFlash2 drafting on Metal; MOSS-Image generates images on CUDA via /v1/images/generations; GLM-OCR runs on HIP; the CUDA build of DeepSeek-V4.1 adds structured output, and long-prompt prefill is now executed in chunks without interrupting responses already in progress; LFM2-MoE, LFM3, MiniMax-M3, and Qwen3.5-2B can interleave and fill in new prompts on Metal. On the server side, there are per-request deadlines, a prompt-cache flush endpoint, and deferred/namespaced tools.

Image accompanying the TensorFold 1.3.3 release tweet

But viewed from another angle, this scorecard is a different ledger. X user @DonaldJ.Gorbachev titled his repost 'The five-second epistemology of TensorFlop 1.3.3,' casually rendering TensorFold as TensorFlop, essentially saying: it takes five seconds to see through the tricks in this release.

He ran several calculations:

First, hardware cost. Four GB10s are four DGX Sparks. At the $7,900 list price, the hosts alone cost $31,600, not counting the 200G cables. The bandwidth this combo buys is 1.09 TB/s of LPDDR5X.

Second, the prefill stopwatch. Prefill is the stage in which the model reads the entire prompt, computes the intermediate state, and then begins generating the first token. Ash's benchmark reports median drafted decode after a 1,040,384-token prompt. Donald points out that this number does not answer a more basic question: how long did prefill run on the four machines before they emitted the first token? His reference figure is that a 1M prefill on 2x Spark takes 13.6 minutes. If four Sparks show no meaningful improvement at this stage, then the most eye-catching decode number in the release post is merely the speed at the tail end of long text.

Third, 124 PRs in 24 hours. That averages one merge every 11.6 minutes, a pace that looks more like automated pipeline output than thorough review. Donald added one more point: DeepSeek-V4.1 only now adds structured output and chunked prefill, so its basic capabilities are not being filled in very quickly.

Image accompanying @DonaldJ.Gorbachev's tweet

To make his point, he casually compared it with the Wave 10 system in his kitchen. Five AMD MI50s (128 GiB HBM2) plus two Blackhole P150As (64 GiB GDDR6, plus 330 MiB on-die Tensix L1 SRAM), interconnected by four 800G FABRIC_1D cables. The total price is under $10,000, and off-chip bandwidth is 6.14 TB/s, 5.6x that of the four-Spark combo. He says this system can run the 763B DeepSeek-V4.1-Flash-1017, with all 170/170 gfx906 kernels zero-scratch and no Python; cold NVMe .kvblk restores a 780K context in 357 ms, and it has been validated on a real 132-step C++17 build. The relevant notes are on GitHub.

This debate is not simply a dispute over fast versus slow. Ash gives the prettiest stretch of speed in the release announcement, while Donald asks about costs, prefill, and other stages no one showcases. Benchmarks for AI inference have never been only about whether the numbers are true or false; they also depend on which segment you choose as the selling point. If the next release publishes the 1M prefill stopwatch, there will be far less debate.

Original post: https://x.com/donaldgorbachev/status/2124255546056614286
Release post: https://x.com/ashxhart/status/2124248353722474508
Wave 10 system notes: https://github.com/bespokeontology/dsv41-wht128-hlut2/releases/tag/v2.1.0-wave10-192g

发布时间: 2026-11-22 06:33