On a MacBook, a 9B model goes from 42 seconds to 11 seconds—can the 4x speedup be reproduced?

Akshay runs Qwen3.5 9B on an M5 MacBook Pro. For the same prompt, three decoding methods: 42 seconds without speculation, 30 seconds with regular speculation, and 11 seconds with DFlash+Weaver. Akshay says this amounts to about a 4x speedup and is 2.6x faster than regular speculative decoding. The video shows the three methods side by side.

Speculative decoding is not complicated: a small model first guesses the next few tokens, and the main model verifies all the candidates in one pass. Fast drafters like DFlash guess each position in parallel, but the problem is that each guess is independent of the others—position 5 cannot see what position 4 chose, so if an earlier guess is wrong, all later ones are discarded.

Weaver supplies exactly this dependency relationship. It is a Transformer adapter layered on top of DFlash that selects from DFlash's top candidates from left to right, with each choice depending on the previous one; it grows a tree of possible continuations, and the main model verifies the entire tree in one pass. On Akshay's Mac, each model pass produces 7.5 tokens, compared with only 1 for the regular method. On a B200 GPU, the paper Akshay cites reports 4x faster decoding for chat, math, and code.

TryMirai open-sourced the entire stack in the uzu engine, targeting Apple Silicon. The GitHub repository is trymirai/uzu.

Akshay gave two commands:

brew install mirai
mirai --model trymirai/Qwen3.5-9B-M -m "Write a python function to merge two sorted lists"

The comments section did not offer unanimous applause. Kane Jin reproduced it on an M2 MacBook Air 24GB with Qwen3.5 9B, temperature 0, and 512 output tokens; after warm-up, the median was uzu 15.1 tok/s, MLX 14.8, llama.cpp 12.9, and Ollama 8.3. Uzu reported 512 output tokens across 513 verification passes, about 1 token/pass. He asked whether there was a configuration step where speculation was not enabled, or whether the speedup mainly only appears on M5.

The Broke Vibe Coder asked for the variables to be separated: uzu with speculation off, DFlash only, and DFlash+Weaver; otherwise a 4x result cannot clarify how much Weaver contributes on its own. Nodelog reminded that speed depends heavily on draft acceptance rate, and that one should log how many tokens are accepted per step on one's own prompts, not just look at wall-clock time; also compare with batch size greater than 1, because gains shrink once the target model becomes compute-bound. Tom Sheffield wanted to see long multi-step tasks with continuously growing context; Aniket Tapre asked whether draft drift can still be maintained in long code outputs; Wecko AI asked whether it can double on a 27B model. The AI Engine Room was concerned about how much extra memory the Weaver adapter takes when placed next to the 9B model on a Mac.

On quality, Akesar's explanation is that with standard speculative decoding verification, answer quality should not degrade: the main model still checks every guessed token and discards those it would not have chosen; only speed changes. This explanation holds provided that the verification logic is correct and the acceptance rate is high enough.

Anirudh's complaint was more direct: after a 4x speedup, agents can confidently make mistakes faster—at least there is no API bill to blame. Yuzu added another jab: proving a massive breakthrough with a function that merges two sorted lists. Actian for Developers saw the other side: after 11 seconds, the only slow point left in local RAG may be retrieval.

More importantly, results like this are usually not a universal promise. A single prompt, a single machine, and a specific quantization format all change the acceptance rate. The M2 Air data already shows that being able to install an open-source stack does not mean a 4x speedup comes automatically. uzu is worth trying, but first record acceptance rate, long-context performance, and long code outputs, then decide whether to write 4x into your own expectations.