“All Code Is Slop (Except Mine)”: 11,000 PRs in a Month, 0.5% Rollback Rate

The AI Engineer channel has released the recording of Lauren Tan's (@poteto) conference talk, titled “All Code Is Slop (Except Mine).” It runs 17 and a half minutes and is about how her Grok Bot team at SpaceXSI uses AI coding agents to ship quickly while keeping the rollback rate around 0.5%.

Let's start with the numbers: 11,000 pull requests in one month, with a rollback rate of about 0.5%. For comparison, many teams may not even hit that number with human-written code.

“All code is slop” is self-deprecation in the talk title. It means that in a context where AI generates huge amounts of code, everyone assumes other people's code is flawed, while only their own part is trustworthy. This bias is even more obvious with AI agents: people naturally distrust machine-generated code. The question Lauren wants to answer is how to turn “slop” into something shippable.

Her answer isn't complicated, but executing it is heavy.

First, the trust pyramid. Code quality is divided into several layers from bottom to top: at the bottom are architectural constraints; above them are lint rules, type checking, unit tests, integration tests, runtime validation, and formal verification. The higher you go, the higher the cost of a single validation, but the more critical the edge cases covered. Many teams only do the bottom two layers and then rush to have AI agents submit PRs like crazy; the result is an exploding review queue and a persistently high rollback rate.

Dune is an internal tool of theirs; its core idea is to turn architectural rules into executable checks. Dependency directions between modules, layering boundaries, which packages cannot import which other packages—all of it is written as machine-verifiable rules. Human memory is unreliable, and comments can go stale, but if it doesn't pass in CI, it doesn't pass.

There's another detail: they use lint rules to replace a large number of comments. Comments tell later readers “it should be like this here”; lint rules directly tell the machine “if it's not like this, it doesn't get merged.” AI agents won't read comments, but they will run lint.

Runtime validation and feature maps solve another class of problems. Code compiling and tests passing doesn't mean runtime behavior is correct. The feature map marks the runtime paths of key functionality; after an agent makes changes, it automatically runs the corresponding validation paths.

Formal verification uses TLA+ and Lean. This part sounds heavy, but it makes sense under the framework of “spending tokens to buy certainty.” Lauren's point is that tokens are cheap now, and spending tokens on verification is more cost-effective than spending them on generation. Generating a piece of code takes only seconds; verifying whether it broke the system may take tens of minutes of CI, but compared with production incidents and manual rollbacks, the math works out.

Cursor agents and pstack are execution-layer pieces. Agents take tasks, modify code, run validation, and open PRs; the whole process is automated. But the order Lauren repeatedly emphasizes is: first build a solid software engineering foundation, then talk about a software factory. If the foundation isn't solid, the faster the agent runs, the faster garbage is produced.

There's a judgment in the talk worth noting: the bottleneck for AI coding agents is not generation speed, but verification throughput. The generation side is already fast enough; if the verification side can't keep up, the whole pipeline gets stuck at review and rollback. 11,000 PRs paired with a 0.5% rollback rate shows they got verification throughput up.

The video is below; 17 minutes, with high information density.

Speaker info:

• X: https://x.com/poteto
• GitHub: https://github.com/poteto
• pstack: https://github.com/cursor/plugins/tree/main/pstack