qwenGPU: Running CUDA on 45 Untrained Layers of Qwen3.5-2B

MarkTechPost introduces an experiment named qwenGPU: it compiles a 4-way SIMT GPU into Qwen3.5-2B, implemented as 45 untrained layers. The model generates steps for each real CUDA launch on its own, and uses Gated DeltaNet states to serve as registers and memory.

First, some background context: CUDA is NVIDIA's parallel computing interface, SIMT stands for Single Instruction Multiple Thread, and GPUs process tasks in parallel via a large set of compute lanes. qwenGPU only implements 4 lanes, a scale so small it acts like a teaching demo. Its goal does not seem to prioritize throughput, but to verify a core possibility: can GPU execution semantics be embedded directly into the forward pass computation of a large language model.

The phrase "runs CUDA" in the headline is easily misleading. Per public descriptions, it does not port NVIDIA drivers or an actual physical graphics card into the model. More accurately, 45 extra layers are added inside the model, and these layers act as an ultra-lightweight SIMT executor, where the full CUDA launch flow unfolds step by step within the model. The so-called "model writes every step by itself" covers operations including kernel launch configuration generation, parameter passing, memory read and write. The CUDA launch process is inherently complicated: grids, blocks, shared memory and synchronization points are all common sources of errors at every step. Letting the model write these steps means delegating the entire execution plan to the LLM's forward computation.

Gated DeltaNet states are critical to this design. Gated DeltaNet is a gated linear attention / state model that preserves historical information via recursive states. qwenGPU repurposes these states as registers and memory, which lets the model maintain the full GPU execution context as sequence inference progresses. This approach is quite clever, but its limitations are also obvious: the state dimension defines the maximum addressable space, 4 lanes cap the parallelism, and 45 layers set the upper limit for program complexity that can be unfolded. The fact that it can run CUDA does not mean it runs fast, let alone that it can execute large kernels.

The 45 untrained layers are another notable detail. Literally, these layers are not fitted to GPU behaviors with training data, but are defined directly through the compilation process. Public summaries do not elaborate on their initialization method, nor specify which subset of CUDA is supported. With only the available introduction, there is no way to confirm if it can handle common structures in real world kernels such as branches, loops and atomic operations. What is confirmed is that the researchers took a highly unconventional path: they compiled execution rules directly into model weights and layer structures, skipping the common practice of fine-tuning models with CUDA datasets.

The most valuable part of this work lies in the core question it explores: if a model can maintain an executable GPU state, it does not only generate code, but can also run the full execution flow internally. For AI practitioners, this provides a new line of thought for embedding program semantics into model architectures; for general readers, it can be understood as building a tiny GPU simulator inside a large language model. It is still very far from practical deployment, but it is already highly interesting as an architecture probe.

Original link: MarkTechPost