Changing Just 0.016% of Weights Takes Math from 7.9% to 48.1%: LLMs Begin Directly Reading and Writing Weights

The tweet comes from the DSPy account and is a thread by Rikiya Takehi. The research team had GPT-6 Astra edit the weights of Qwen3.5-0.8B, with the goal of improving its math ability. First, the numbers: only 0.016% of the weights were changed, MATH-500 went from 7.9% to 48.1%, and the IFEval score tripled. No gradients, no training.

For an 0.8B model, counting 800 million parameters, 0.016% is roughly 128,000 weights. The change is very small, yet it fixed the model’s reasoning loop. According to the tweet, GPT-6 Astra can directly read and modify neural network weights; the authors call this approach Weight-Space Programming.

Sparks of Weight-Space Programming: LLMs can natively read & write neural network weights

For general readers, weights can be understood as the model’s internal adjustable parameters. The usual approach is to train on data and update with gradients. The claim here is that a frontier LLM finds and rewrites weights directly, like editing a piece of structured data. It touches on two long-standing problems at once: optimization and interpretability.

The tweet also mentions that GPT-6 Astra can identify the task of an already-trained network without environment feedback, and generate a game network stronger than one trained with PPO for 100 million steps. This part is more of a capability demonstration; for details, see the blog. The blog is credited to Rikiya Takehi, Jayden Teoh, Alex Zhang, and Omar Khattab, from MIT CSAIL.

Blog and tweet links:

At present, these numbers come from the researchers’ public description. Code, reproduction conditions, and evaluation details should be based on the blog and subsequent discussion. The key number is 0.016%. The 6x is just the result.