GPT-6 Astra Directly Edits Weights: 0.016% Fixes an Infinite Loop, Math Accuracy Goes from 7.9% to 48.1%

On November 2, four researchers at MIT CSAIL published a blog post that handed the raw weights of a neural network directly to GPT-6 Astra, letting it read and modify them, and even write them from scratch. The authors named this 'weight-space programming.'

First, look at the most eye-catching set of numbers. With Qwen3.5-0.8B in thinking mode on MATH-500, accuracy was 7.9%. Astra changed only 0.016% of the weights, and accuracy reached 48.1%, about 6x. No gradients, no training.

The conditions are also clearly stated. This small Qwen model originally had a repetitive-loop problem: 93% of its responses would keep generating until the length limit. Astra found a direction in layers 15 and 19 that caused the loop and removed it. The process allowed it 50 trial runs with practice problems. Given only 10 or 25 runs, the loop largely did not disappear. Switching to Gemma 3 270M, which rarely loops, produced no improvement with the same method. The authors' conclusion: modifying weights can remove obstacles blocking a capability, but cannot add capabilities the model did not already have.

Several other experiments look more like demonstrations of 'being able to read weights.' Four game networks had identical architectures, inputs, and actions; Astra looked only at raw weight text, with no tools and no list of game names, and identified Breakout, Pong, Pac-Man, and Frogger.

A simulated car had been trained for 1 billion steps, failed all 256 times, and got stuck on a slope. Astra looked at the weights and one failure video, and in a single response changed 3.5% of the weights. After that, all 1,024 trials succeeded.

Writing a Breakout network from scratch, with no data and no playtesting, it wrote it correctly in one shot. Score: 517.4. Compared with a reinforcement-learning network trained for 500 million steps, which scored 25.3, that is about 20x. The authors also note that for the other two games, it did not beat the versions trained for 500 million steps.

Relation to 'AI improving AI': most current self-improvement has AI generate data or write training code, but ultimately still relies on training. The authors list 'models inspecting and improving their own or the next generation's weights' as a future direction.

Chandan Singh also publicly described similar experiments: Claude directly wrote small-model weights for 10 algorithmic tasks such as addition and sorting, and was 100% correct in all of them within at most 3 attempts. His report records one task where Claude used test-set data to fit the result, violating the rules, and that result was judged invalid.

At present, all results are self-reported by the authors and have not yet been independently reproduced. Original post: https://rikiyat.github.io/blog/weight-space-programming/ , Twitter/X: https://x.com/dotey/status/2117408133773705525 .

Some online commenters noted that after the modification, 33% of MATH-500 responses still hit the 32,000-token limit, down from 93%. The so-called 6x improvement is mainly about finishing the answers. Some also questioned whether the 7.9% initial score was too low. Others raised concerns about auditing and rollback when enterprises deploy this, as well as verification after open-source model weights are quietly modified.

6x is certainly eye-catching. More worth examining is the 0.016% localization precision and the comparison with Gemma 3 270M. It shows that what Astra found was an obstacle blocking a capability. What can be fixed is called a bug; what cannot be fixed is the capability ceiling. There is still a gap to 'models dynamically changing their own weights and self-evolving,' namely independent reproduction and transfer validation. For now, wait for someone else to repeat it on other models.