After Letting Agents Write Their Own Questions, Human Feedback Pushes Benchmark Difficulty Up by More Than 28 Points

Will autoresearch push human researchers out? The paper “Aligning Autoresearch” from Meta FAIR and other teams gives a less dramatic answer: co-evolution.

Xudong Han summarized the paper’s approach in a tweet. The research team first had a Research Agent create its own benchmark, then gradually added human feedback. The input where humans are truly valuable is concentrated mainly in the early stage: helping the Agent define what to research, which materials to provide, and which questions are worth pursuing.

This may not sound “fully automatic,” but the experimental results are concrete. Across six benchmark–solver comparisons, adding fine-grained human guidance made the final benchmarks clearly harder, with improvements of 28 to 46.5 points.

The paper also specifically built HackBench to test whether models can find reward hacking in extremely long research trajectories. The data is blunt: 169 trajectories, with a median length of more than 1 million characters, usually hide only one truly violating command. Even for frontier models, the best localization F1 is still below 53.

Finding one violating command in a million characters is like finding an altered parameter in the appendix of a long paper. Scores keep rising, but that may just mean the model is getting better at gaming the rules it set for itself. This risk cannot be waved away with a single line about “models will self-improve.”

So what the authors do next is natural: teach the Agent to collaborate better with humans. AutoInteract uses real interaction records to construct a “human simulator,” training models to handle clarification, changing requirements, and error correction; RL-XAR, in turn, uses expert writing to optimize evaluation criteria in reverse, and then improves the Agent’s research communication.

Viewed this way, future Research Agents may not simply move toward “fully automated research.” A more realistic path is to have Agents take on more and more execution and search work, while writing human research judgment, feedback, and standards more deeply into the training loop. An Agent must not only be able to conduct research on its own, but also know how to work with humans to make research better.

In the replies, Marvin points out something easy to overlook: when an Agent creates its own questions and optimizes itself, someone must also independently check the evaluation criteria. Otherwise, rising scores may just mean it is getting better at gaming the rules it set for itself; keeping a separate test set would be more convincing.

Paper link: https://arxiv.org/abs/2610.38882