Changing Just 0.016% of the Weights, a 0.8B Model’s Math Performance Reaches Up to 6×: LLMs Begin Directly Editing Weights

By changing only 0.016% of the weights, a 0.8B model’s math performance rose to as much as 6× its original level, and instruction following to 3×. No gradients, no training, and no rerunning the dataset. The experiment Rikiya Takehi released on November 2 turns an LLM directly editing another model’s weights from metaphor into operation.

Neuroscience has studied the human brain for a hundred years, yet access remains poor: it can record only a few thousand neurons at a time, cannot read synaptic weights, cannot copy the brain, and cannot undo interventions. Neural networks are the opposite: every weight is readable, every modification can be rolled back, and the entire state can be copied. But to humans, a matrix with 0.8B parameters is as hard to understand as a sheet of cortex. Fully observable, almost incomprehensible.

The opening in this experiment is exactly there: humans can’t read it, so let another model read and write it. Rikiya calls the method Weight-Space Programming, which means quite directly that frontier LLMs can natively read and write neural network weights. Both the original tweet and the blog point in the same direction: the model treats raw weights as readable, editable code, diagnoses failures, and then directly patches them.

Reading weights

Astra was given the raw weights of four game policies, one at a time. The four networks had identical architectures, inputs, and actions, and it was given no tools and no list of games. It correctly named the four games: Breakout, Pong, Pac-Man, and Freeway.

Fixing weights

A PPO car policy kept getting stuck on a slope. Given the weights, the scene, and one failed trajectory, Astra wrote a patch in a single reply without using tools. After changing 3.5% of the weights, the success rate reached 100%.

Writing weights from scratch

Given the rules and code for a MinAtar game, Astra wrote the weights for a Breakout network in a single reply. No training, no tools. On MinAtar Breakout it averaged 517.4 points; PPO trained for 500M steps gets 25.3, about 20× less.

Editing another LLM

More noteworthy is editing Qwen3.5-0.8B. Astra was asked to make this small model better at math. It saw that Qwen fell into a reasoning loop, a problem Qwen’s own model card also warned about. With a budget of 50 forward passes on practice problems, Astra located the cause and changed it directly, with no gradients and no training.

Only 0.016% of the weights were touched, and math performance rose to as much as 6× the original, with instruction following to 3×. The gains from repairing the reasoning loop also spilled over beyond math.

Where this result fits

Rikiya’s blog is clear that they are building a benchmark to evaluate this and also building a harness to help LLMs explore and write weights. In its retelling, AGTP attributes the work to MIT researchers including Rikiya Takehi and lists the key numbers: 0.016% of weights directly modified, math up to 6×, instruction following 3×, and a 50-forward-pass budget.

In the comments, some focused on mechanistic interpretability: if this really generalizes, it would be a revolutionary finding and a prerequisite for scalable interpretability. Others cautioned that being able to read all the weights and being able to understand them are completely different things. Michael Waitze put it more bluntly: the weights are fully readable and writable, but explanation remains the hard part. Another researcher said the results have not yet been verified and they are unsure whether the method can scale, but they expect something similar to appear within a year. Someone also asked what data OpenAI fed Astra.

A safer judgment is that this is still just a 0.8B model and a few small experiments, and cannot be extrapolated to all models and all tasks. But it bypasses humans reading weights layer by layer and directly treats another model as an editable object. This action itself is more worth watching than a one-off performance gain, because it changes the entry point for optimization actions.

Original tweet: https://x.com/rikiyatakehi/status/2117313742891327994

Blog: https://rikiyat.github.io/blog/weight-space-programming/