Will Zero-Data Self-Improving Models Secretly Pick the Easy Problems?

Xiaomi Research released the paper A-RSI on October 27, and Hugging Face marked it as the second most popular paper of the day. The key term in the title is zero-data multimodal self-improvement: without relying on external labels, the model generates its own tasks, verifies its own answers, and then makes itself stronger.

The mechanism is not complicated. During training, it monitors two metrics: reward change and accuracy. When the learning signal starts to decline, it means the current task pool is almost exhausted, so the system generates a new batch of tasks to refresh the pool. How new tasks are selected, how refresh rules are set, and whether answers are correct are all handled through programmatic verification throughout, without human annotation and without touching test-set scores.

On 11 major benchmark suites, A-RSI used 800 newly generated tasks and raised the average score from 57.77 to 61.05. The numbers are not explosive, but the approach is interesting: where the data comes from, when to replace it, and how to confirm answer reliability after replacement are three questions crammed into one loop for solving.

Paper screenshot:

Some people in the comments praised it, saying that refreshing the task pool when the learning signal declines is a clean way to handle it. But the two comments really worth reading are these:

One asks, what stops it from gradually picking easier tasks for itself? It is followed by a self-deprecating line saying that if it were them, they would definitely do the same.

The other asks, zero-data self-improvement sounds great, until you ask what it uses to score itself. What stops it from drifting?

These two questions are two sides of the same coin. Adaptive refreshing assumes that the learning signal itself is trustworthy. If the model has a way to make things more comfortable for itself during task selection or answer generation, this closed loop could slide from self-improvement into self-soothing. The paper says it automatically verifies the selected attributes and refresh rules, and separately audits the generated answers. This design makes sense from an engineering perspective. But the auditors and the audited modules are both inside the system, and the paper does not explain where the boundary is drawn in enough detail to reassure everyone.

The hard part of this kind of self-improvement framework has never been whether it can raise scores. Raising scores is easy; what is hard is whether you can still trust it after the scores go up. 57.77 to 61.05 is a nice curve. But if there is a hidden line beneath the curve where tasks quietly become simpler, then it is not self-improvement; it is letting itself off the hook.