He installed a GPU inside Qwen's weights, runs real CUDA, 10 tokens per second

Vixhal (@TheVixhal) stuffed a GPU into Qwen3.5-2B.

More precisely, he compiled a 4-lane SIMT GPU into 45 layers of untrained weights and placed them in front of Qwen's original 24 layers. 69 layers total. It runs real CUDA, compiles to PTX, and executes inside the model's own forward pass.

Previously Qwenputer compiled a RISC-V CPU into Qwen3.5-4B, and Percepta compiled a WebAssembly interpreter into a custom transformer. qwenGPU's difference is where the memory comes from: it uses Gated DeltaNet's recurrent state as registers, program ROM, and global memory.

Gated DeltaNet is a kind of linear attention, where each head maintains a state matrix. Normally gating controls writes, and keys and values perform an outer-product update. Vixhal's approach is to keep the gate always open, set the write strength to 1, and make all keys one-hot. This way each update overwrites exactly one row, and when read back with a one-hot query, that row is an addressable memory cell.

The configuration is 4 lanes, 15 usable registers per lane (one of the 16 rows is always read as zero), a 4-stage pipeline, 32 blocks of ROM, and 126 memory words.

The CUDA launch flow: the host plays the role of NVRTC and the CUDA driver, using clang to compile CUDA into PTX, then lowering it to machine blocks, which are fed to the model as tokens. After that, the model writes each step of the launch itself. Each EXEC(b, w) token runs one warp of one block and takes one forward pass. The following VAL tokens carry register results, and MEM records memory writes. The host is only responsible for reading the results back from the recurrent state; it does no computation.

In the demo, the dot product of [1, 2, 3] and [4, 5, 6] returns 32. The model used 47 tokens to finish writing this launch.

On the numbers side, the reference interpreter matches a real Tesla T4 on all 49 kernels. Text drift KL is about 7e-4, because the circuit is silent on text tokens, and Qwen's own weights were not changed. In normal mode, 20 of 24 chat questions used the GPU to compute a number; in advanced mode, Qwen writes CUDA itself and succeeded on the first try 4 out of 5 times.

The cost is also straightforward. On Apple MPS, 10 to 15 tokens per second. Building the checkpoint requires about 16GB of free memory, there is no precompiled version, and you have to just build it yourself. The code is MIT, and the checkpoint is Apache-2.0 like Qwen.

Someone asked whether it could run a model inside the model. Vixhal's reply was that technically yes, but it would have to fit into 126 integer words. A 2-2-1 XOR network is no problem; running Qwen would require a much larger GPU.

Someone else did the math. Marco Polo complained in a reply: your host script is already running Python; instead of importing decimal, you compiled 45 layers of 1970s C into 14.7GB of VRAM, multiplying three integers takes 20 seconds, and when it encounters $0.50 it reports “GPUError: use integers.” He also attached his own 70-Cell Pipeline benchmark.

Vixhal didn't take the bait and replied with a screenshot of NVIDIA falling 4.22% that day.

Is this thing practical? The answer is not very optimistic. At 10 to 15 tokens per second, running a piece of CUDA is not much faster than doing it by hand on paper. The CUDA Qwen writes itself is also only 80% correct, and the README itself warns that CUDA from a 2B model is unreliable.

But the 49/49 matching T4 result shows one thing: precisely executing arbitrary programs inside the weights of a pretrained LLM is feasible. The model is not just approximating a function; it can perform deterministic, verifiable computation within its own reasoning process. Percepta and Qwenputer are both moving in this direction, and qwenGPU changes the granularity from CPU instructions to GPU kernels.

As for where this road leads, it's too early to conclude. 49/49 is only a starting point; speed is another problem. At 10 to 15 tokens per second, for now it's not suitable as an inference backend. But it at least proves one thing: the things you can stuff into pretrained weights are more than most people think.