Wink Pings

5,000 PRs and a $250,000 AI Bill: CodeRabbit Founder Says Human Code Review Is Dead

Harjot Gill spent $250,000 on AI agents last month, and the company merged more than 5,000 PRs. On the Pragmatic Engineer podcast, he explains where the money went and why engineers no longer review AI-written code line by line, but instead review plans and releases.

Start with the numbers. CodeRabbit founder Harjot Gill spent $250,000 on AI agents last month, and the company merged more than 5,000 PRs. These two figures come from his recent conversation on Gergely Orosz's Pragmatic Engineer podcast.

CodeRabbit makes AI code review, and its customers are development teams. It is itself a heavy user of AI writing code and AI reviewing code. Gergely listed three key points from the conversation in a tweet; below I expand on them as I understand them.

$250,000 sounds scary, but at a company like CodeRabbit, it's R&D budget

Gergely's judgment is that CodeRabbit's AI spending is more like R&D, because the new solutions it produces can be productized and sold. For a company with a fixed customer base and an unchanged product shape, the math doesn't work.

Harjot gave a more concrete example on the show. They asked a vendor for a CI proposal, and the vendor quoted $13 million per year. He decided to build it himself with agents, finished it in a week, at 50% lower cost than the quote and with similar performance. This example is more persuasive than the slogan "AI saves money" because it has a quote, a timeline, and a comparison.

Humans no longer review AI-written code line by line; they review plans and final outputs

According to Harjot, engineers inside CodeRabbit review releases—that is, the result after multiple PRs are bundled together—not individual PRs. Gergely summarized this as "Human code review is dead, long live the human review."

This shift makes sense. At 5,000 PRs a month, averaging more than 160 a day, line-by-line review is simply unrealistic. Someone in the comments pointed out directly that reading diffs has already become a full-time job. Moving review earlier to plans and later to releases, with eval and automation in between, is the only way to scale.

But trust is a problem. Another commenter said the hard part is getting people to trust a release they haven't read line by line. That concern is not unreasonable.

Hiring for agency, breadth of experience, and systems thinking

Harjot posted an unconventional job ad: having submitted more than 1,000 non-spam PRs in the past 30 days, or having played 500+ hours of strategy and management games such as Civilization, Age of Empires, Transport Tycoon, and Factorio. His point was that running dozens of agents at once and watching dozens of units in a strategy game have similarities he can't pretend not to see.

The ad was retweeted quite a few times on Twitter. Some thought it was marketing; others discussed it seriously. Gergely included it among his three learning points, which suggests he doesn't think it's pure gimmick.

Two questions in the comments that the episode may not fully answer

Royce Myers raised the attribution question: of those 5,000 PRs, which ones actually earned back the token money? Without cost accounting for each agent, $250,000 is just a murky number.

Debayan asked about cross-version regressions: when agents push hundreds of PRs a day, who catches the bugs that only appear after multiple releases stack up?

Harjot's answer is in the podcast, and the direction is roughly eval, formal methods, and release-level review. How exactly to implement it requires listening to the full episode.

The episode can be heard on YouTube, Spotify, and Apple Podcasts; a text summary is in Pragmatic Engineer's newsletter.

What's worth noting is not the number $250,000, but the organizational form it corresponds to: plans are set by humans, code is written by agents, and review moves from the line level to the release level. This process works at CodeRabbit, but the premise is that what it sells is a review tool. If another company copies it, first figure out where its own $13 million quote is.

发布时间: 2026-11-19 04:29