120B model trained into an organization: RLCR technical report goes live, agents learn to recruit verifiers

The RLCR technical report is now live on Papers with Code. Niels Rogge posted the link on X; the report is titled Avalon-M1: Reinforcement Learning from Collective Rewards, with authors including Iliyan Zarov and others.

RLCR stands for Reinforcement Learning from Collective Rewards. General Reasoning's description is very direct: it is a method for training language models into agent organisations, where multiple agents share a message board and the same workspace. The team used it to train Avalon-M1-Mini, with 120B parameters, and observed the model's behavior within the organization.

The public results cover three fairly concrete directions. The model learned organizational hierarchy and division of labor; collective verification behavior increased, with actions such as recruiting verifiers and testers appearing; and so-called collective thinking emerged, with agents considering other agents more and how to help them.

Taken together, RLCR places the training objective on organizational behavior: how agents divide work, verify, and help one another, while an individual agent's answer score is only part of it. The common multi-agent approach of 'putting models in a group chat to talk to each other' is replaced here by a trainable organizational structure.

Video accompanying the tweet:

General Reasoning believes RLCR can serve as a laboratory for studying collective intelligence and multi-agent safety. That judgment has a basis. Agents recruit verifiers and testers, form hierarchies, and help one another, so the object of evaluation is no longer just a single model's output, but also organizational processes, verification chains, and modes of collaboration. Some difficult-to-test problems in multi-agent safety will surface precisely in such processes.

Netizens' reactions also focused on this type of question. Some asked where the weights are, some were curious whether such models could be used for automated scientific research, and other comments mentioned whether, if trained properly, small teams are actually more effective than large swarms. The material does not provide information on whether the weights are open.

Technical report link: https://paperswithcode.co/paper/131423

General Reasoning original tweet: https://x.com/GenReasoning/status/2113590771722952908

Niels Rogge's tweet: https://x.com/NielsRogge/status/2115020119491448991