GPT-6 Changed Qwen's Weights, and 0.016% Cured Its Infinite Loop

Four researchers at MIT CSAIL—Rikiya Takehi, Jayden Teoh, Alex Zhang, and Omar Khattab—published a blog post titled “Sparks of Weight-Space Programming.” The core conclusion in one sentence: frontier LLMs can now directly read and write neural network weights.

They call this Weight-Space Programming. No gradient descent, no training loop; the model directly uses natural-language reasoning to figure out what each weight block is computing, then makes changes.

The most attention-grabbing result was on Qwen3.5-0.8B. This small 800-million-parameter model had a problem with math problems: its reasoning would get stuck in a loop, repeatedly outputting the same passage, and could go almost all the way to 32,768 tokens without escaping. Qwen’s own model card had warned about this issue.

After GPT-6 Astra got the weights, it found the source of the loop. It localized a direction written by the MLPs in layers 15 and 19, then subtracted that direction from each layer’s 64 output columns. In total, it changed 131,072 weights, or 0.016% of the model.

Results: MATH-500 rose from 7.9% to 48.1%, GSM8K from 13.8% to 67.1%, and IFEval from 21.0% to 62.3%. Same sampling settings, no retraining.

Even more surprising was the gain on IFEval. The looping problem clearly did not distinguish between math problems and instruction-following tasks; once the loop was removed, both improved.

There are several other experiments in the same blog post worth looking at.

They trained four PPO networks to play four grid games, with identical input channels, action spaces, and network architectures—only the weights differed. They fed 17,253 weights directly to Astra as text, without giving the game names, and asked it to guess. It got all four right.

The second experiment switched to diffusion models and asked Astra what shape each model generated. From the output-layer weights, it read out the edge outlines of circles, triangles, and squares. Interestingly, it drew hollow outlines, while the actual models generate solid shapes. This mistake actually suggests it was not running a forward pass in its head, but reading the structure of the weights themselves.

The third experiment was even more extreme. They removed the input and output layers of a small language model and gave Astra only the weights of the middle four transformer layers, with no character labels and no ability to run the model. They asked whether its training corpus was Python, Romanized Japanese, or English. It got all three right. It also independently found all the copying heads (induction heads), exactly matching the results measured by running the model. Elhage et al. proposed this localization method in 2021, and the research team never told Astra about it.

On the weight-fixing side, in Kinetix there was a car stuck on a small slope. After 1 billion PPO training steps, it converged to a local optimum, drove straight into the slope, and got stuck; it failed all 256 tests. Astra modified 3.5% of the weights and added a very thin bypass that reads the car’s speed and position, letting the car reverse a bit and then dash forward. It succeeded on all 1,024 tests.

They also tried writing weights from scratch. For three MinAtar games, Astra received only the rules, source code, and network architecture—no data, no gradients, no tools. The network it wrote scored 517.4 on Breakout; PPO trained for 100 million steps scored 15.8, and for 500 million steps scored 25.3. On Asterix and Seaquest, it approached or matched the level of PPO trained for 500 million steps. The strategy it wrote was not mysterious either: the first layer computes where the ball will land, and the second layer moves the paddle there.

The Go experiment is even more vivid. With 1,480,274 weights, Astra wrote them out line by line in a Python sandbox: first mark stones, liberties, and empty points; then propagate a few steps along stone chains to find dangerous groups; finally score each move. With zero forward passes, it won more than half its games against two small PPO networks. After being allowed 50 forward passes, it began playing practice games against itself, seeing where it lost, and changing the corresponding weights; in the end, it tied or even surpassed a same-sized PPO trained for 20 million games.

There were failure cases too.

Experiment 7 tried to make it distinguish two fine-tuned Gemma 3 270M models: one told the truth, and one flattered. Looking only at the weights, it failed all four attempts. Astra listed many weight differences but could not translate them into behavioral differences. Later, under looser conditions, it was allowed to write 16 passages of text itself and look only at the attention patterns of the two models; it picked correctly for all three model pairs. The flattering model’s head 10.2 kept looking at the answer provided by the user, with an 85.7% weight share; in the honest model it was only 0.75%.

There was also an additional experiment that showed more clearly where the boundary lies. Gemma 3 270M rarely looped; only 8.5% of its answers hit the token limit. After Astra read the scorecard, it found that the errors were ordinary arithmetic and algebra step problems, with no single module that could be removed. It tried fine-tuning several times, and MATH-500 moved by less than one percentage point at most. The team’s own conclusion: editing weights can remove obstacles in front of a capability, but it cannot add a capability the model does not already have.

The blog ends by listing several possible directions: auditing backdoors in weights without running evaluations, replacing full training with a single weight edit, adding or forgetting capabilities with zero data, and letting models genuinely read and write their own weights to achieve recursive self-improvement. They are building a benchmark and harness to turn this from black magic into reproducible engineering.

The complete logs are at github.com/RikiyaT/lm_understand-logs; they include the input Astra received in each round, the reasoning summaries it emitted, the code it ran, and the outputs.

Original blog: Spark of Weight-Space Programming