Changing Only 0.016% of Weights Takes Qwen Math from 7.9% to 48.1%, with No Training at All

In a repost, Rohan Paul said that experiments by MIT researchers show that GPT-6 Astra can directly edit the weights of another model without changing gradients or running training. The author in the citation chain, Rikiya Takehi, calls this approach Weight-Space Programming.

The most eye-catching set of results: in a Python notebook, Astra was given the raw weights of Qwen3.5-0.8B, plus a test budget of 50 practice math problems. It did not run gradients or train; it changed only 0.016% of the weights and raised Qwen3.5-0.8B's score on MATH-500 from 7.9% to 48.1%. The author says math improves by up to 6x and instruction following by 3x.

The problem Astra saw was very specific: the model would repeatedly repeat itself until it hit the length limit. It traced this habit to an internal signal written by a two-layer network, then subtracted that signal from the weights. Once the loop disappeared, the model could finish problems it already knew how to solve. Based on 0.8B parameters, 0.016% is about 128,000 parameters. After working out this calculation in the comments, one user called it absurd.

This set of experiments covers more than one scenario. Given four game policies that share an architecture, inputs, and actions, Astra read only the raw weight text, with no tools and no list of games, and recognized all four: Breakout, Pong, Pac-Man, and Freeway.

For a PPO car stuck on a slope, given the weights, the scenario, and one failed trajectory, Astra returned a patch, changed 3.5% of the weights, and achieved a 100% success rate.

Even more radical is writing the weights from scratch: given the MinAtar rules and code, Astra wrote out Breakout network weights in a single reply, with zero training and zero tools, scoring 20x higher than a PPO trained for 500M steps.

The author calls this approach Weight-Space Programming, and gives more details in the blog: https://rikiyat.github.io/blog/weight-space-programming/

Two main viewpoints dominated the comments. Some users asked whether the patch that fixes the repetition loop still holds on non-math prompts, and whether it might quietly break other capabilities. Others focused first on the evaluation setup, arguing that an improvement of this magnitude requires first seeing how the eval is constructed. Still others asked whether a model can modify its own weights, and who would audit the changes once made. These questions have not yet been answered by the experiments.

What can be confirmed now is that model weights are beginning to be treated as readable and writable code to be edited. The change is as small as 0.016%, and the result is large enough that people should stop and look at the evaluation first.