Only 217 Real Loops in 36,487 Repositories: Most Agents Are Crons With a Credit Card

36,487 public repositories were screened. The researchers are from Heidelberg and Singapore Management University. They first found 256 repositories suspected of running autonomous agent loops, then inspected them one by one and confirmed 217. That is 0.59%.

The most striking point in the data: almost every repository has a trigger. The things that actually make a loop usable fall off sharply. 101 have spend or round limits, 76 have validators, 55 have goals or stopping rules, 35 save state between runs, and only 10 record cost.

A trigger can be set up in five minutes. A cron, a GitHub Action, a YAML file. The trouble comes after that.

The author, Ichigo, breaks the problem into three words:

• Harness: outside the model, determines what a run can see, touch, and prove.
• Loop: determines whether there is a next run, and what the next run knows that the previous one did not.
• Graph: determines which node runs next, and what each node can see.

These three also nest. The model runs inside the Agent loop, the Agent loop runs inside the work loop, and the work loop runs inside a step of the graph. The everyday word "loop" has at least three layers: the second-level Agent loop, the minute-to-hour work loop, and the day-to-week system loop. Most of the confirmed repositories are triggers wrapped around the first layer. 180 were triggered by repository events, mainly automated PR review. Useful, but not what those long essays depict.

A trigger is not a loop. A trigger is just a cron with a credit card.

A work loop that can converge first needs checks. Agent self-evaluation is unreliable. Anthropic's harness team found that when agents grade their own work, they consistently praise it, even when a human can see at a glance that it is mediocre. So "done" must come from outside the model. Script checks are firmer than a model reading logs. Checks must also satisfy two conditions: they must be able to fail, and the Agent must not be able to change them. Checks that are green before starting prove nothing; if tests can be changed, the model will find the shortest path to green.

Claude Code's /goal judges conditions from the conversation. It does not run commands or read files. It is useful, but the conditions must be provable from the conversation record. Writing "the auth code is correct" is useless; writing "npm test exits 0, existing test files under test/auth are unmodified, git status is clean, stop after 20 rounds" gives a checkable boundary.

Feedback must also be specific. Just saying "test failed" is a verdict; retrying a verdict is like rerolling dice. The next run needs to know which test failed, what was expected, what was actually seen, and where. Output like pytest -q --tb=short is more useful than thousands of lines of logs, which bury the key line.

State must be persisted to disk. 35 repositories saved loop state on the default branch; more did not. Every new run starts from zero, like a shift change where the new engineer does not know what the last shift did. The most cost-saving field in the state file is tried, which records what has been tried so the next run does not walk into the same dead end again. Commenter Levi mentions appending check content and changes each time, then adding a rule: "stop after 3 consecutive no-diff runs"; after a crash, it can still recover from the last line.

Exits must be hard. Fewer than half of the repositories have any limit. There should be at least three exits: checks pass, budget exhausted, or stalled. Budget exhaustion includes rounds and dollars. Stalled means the same failure repeats. Commenter AZRAF says the maximum iteration count is not a real exit; it stops the bill, not the waste. A more reasonable approach is to look at state: if the same error or the same diff appears twice, hand it back to a human. Cost tracking is only 10/217; that number itself says something. Commenter Pascal also reminds us that if loop retries lack an idempotency key, the timeout moment may cause a duplicate charge.

Walls must hold even when no one is watching. "Please don't modify migrations" written in Markdown is a wish; deny rules are walls. Triggers also introduce new risk: a loop reads issues written by strangers while also having code access and push capability, exactly Simon Willison's lethal trifecta. Steps that read untrusted text should not get dangerous tools, and steps with dangerous tools should not directly read raw text.

Finally, there must be a human. Besides green passes, every exit should find a specific person and include the state file. Green is only a proposal; it still needs a human before merging. Boris Cherny ran an experiment letting Claude maintain an app. It opened 388 PRs, and 180 were merged after Claude Code Review and human review. Less than half. Addy Osmani's judgment is more direct: your review bandwidth determines how many loops you can run; tools do not.

Graphs are not a case of more Agents being better. Researchers from Google and MIT ran 180 Agent setups with the same prompts, tools, and budget, changing only the collaboration method. On a financial analysis task that could be cleanly split, an orchestrator plus worker was 81% better than a single Agent. On step-by-step planning tasks, multi-Agent setups were 39% to 70% worse. In parallel with no checks, errors were amplified 17.2x; with an orchestrator catching them in the middle, that dropped to 4.4x. Write routing in code, do checks between steps, and give each step clean context. Commenter Ignacio points out an implementation detail: replacing all numbers in the output with # before fingerprinting would treat "expected 302, saw 200" and "expected 302, saw 500" as the same failure, and the loop would misjudge it as stalled. Timestamps and line numbers can be removed; status codes must be kept.

System loops are slower. They turn one failure into a permanent fix: add a permission, add a lint rule, add a tool, add a line to AGENTS.md. In order: structure first, static checks second, prose only for places that truly need judgment, and human review last. After a model upgrade, outdated harnesses must also be deleted. Anthropic's Prithvi Rajasekaran says every component is a bet on a model shortcoming, and bets expire. Remove a component, rerun the task, and if it is no worse, delete it.

When should you not build a loop? If a script cannot say "done", do not build it. If wrong results are expensive, do not auto-merge. If the task will not recur, one run with checks is enough. If the work cannot be split cleanly, do not add a graph.

The original article is here: Harness, Loops, Graphs, and author Ichigo's tweet is here.

Add the trigger last. First make checks honest, make feedback converge, make state remember, make exits affordable, and make walls hold when no one is there.