The secret RL algorithm CEOs are DMing him to ask him to delete; a single reply in the comments tears it down: this is a perpetual motion machine

Michael R. Bock posted a tweet claiming that the secret algorithm of every current RL environment/data company is the diagram below. He added that this is the RSI algorithm. Then he said that CEOs from many RL environment companies messaged him with roughly the same content: Please delete this, you're leaking the alpha.

A bit of background first. RL environment/data companies provide tasks, environments, and verification signals for reinforcement learning training. RSI usually refers to recursive self-improvement. Bock describes a flowchart as these companies' secret, in a tone that sounds like he's exposing the truth. The comment section did not fall into an 'oh, I see' silence.

Santhosh Subramanian said the method is very naive and looks very expensive. Ben was more direct, calling it a perpetual motion machine. varun pointed out that the diagram is missing 'a highly stressed human in the loop.' Patience Exerciser said they had already implemented 90% and were preparing to improve the process. Yan Gaev asked how to use Opus 5 to generate sufficiently good RL environments or verification tasks, and also asked why Fable 5 is needed. Bock replied that Opus 5.5 is very good. Dominik's question was more specific: once the generator surpasses the verifier, who audits the verifier?

In another reply thread, Andrew Bugera said this is 'gradient descent all the way down.' Someone complained that X is now all slop; Bock replied that he drew the diagram by hand. Someone asked him to explain it with crayons, and he posted another diagram.

A commenter shared their SWE post-training experience: a small model, 6k environments, with reference patch check, tests, and empty patch check. The problem came from specific words in the tests. A valid solution could fail because a certain word was missing from the error message. They said they hadn't yet fed grader feedback back into the rollout for tuning the model, and planned to add it. Bock replied, 'sweet.' This part feels more like the front line than the flowchart: the verifier is not an on/off switch; words, error messages, and how tests are written all affect the signal.

The skepticism continued. Gravity Research asked: if this is the big secret being guarded, what was the $50M valuation story that seed investors heard? Xinming Tu said this pipeline is common knowledge and the alpha is in the generator model. Others guessed the secret is worth zero, someone gave it an 'RSI Slopaward,' and someone said Deeply beta. Nathan Benaich only replied, 'i mean rly?' Fabian threw Bock's own joke back at him: Please delete this, you're leaking the alpha.

Taken together, Bock's tweet is like throwing a flowchart onto the table and saying the secret is here. The repliers each flipped the table over: cost, human-in-the-loop, verifier auditing, generator quality, test details. If a hand-drawn diagram could leak the alpha, then the alpha is probably not in the diagram. What is truly scarce is a team that can consistently produce good environments and good verification signals while also bearing the cost of human-in-the-loop. As for CEOs DMing him to ask for deletion, it might be real, or it might just be part of the tweet. The comment section has already broken down 'too simple to be true' into several not-so-simple engineering problems for readers.

Original tweet: https://x.com/michaelrbock/status/2115819564340773208

Quoted tweet: https://x.com/michaelrbock/status/2115589801491800474