128 sessions queue for 25 seconds; pi-durable cuts it to 0.28 seconds after batching database commits

An agent run writes to the database about 12 times. With 128 sessions running concurrently, each run waits in line for about 25 seconds. After this round of optimization in pi-durable, that wait becomes 0.28 seconds.

Mario Zechner explained on X how pi-durable handles persistence: at every step the agent first saves to the database, and only then continues. 1 save equals 1 commit. The purpose is straightforward: if it crashes halfway through, it recovers from the most recent commit. About 12 commits per run.

The original bottleneck is no mystery. The session sends each commit to the database separately, then waits for it to return. One database round trip is 4 ms. As sessions increase, all commits line up in the same queue. At 128 sessions, each run has to wait about 25 seconds before its turn.

The vertical axis in the second chart is log scale.

The fix: when a commit is in flight, the commits that pile up during that time are sent together in the next database call. With 128 sessions, one db call carries about 33 commits.

Mario says this sounds simple but is actually tricky. Simply batching everything doesn't work, because individual commits may have read/write dependencies. That's where the real work lies: merging commits has to handle ordering and dependencies; you can't just cram a batch of interrelated state into one package.

The tl;dr is also worth reading: if you run a coordinator that fans out to dozens of subagents and the backend is postgres or similar, you no longer need to shard sessions for throughput. Now you're roughly limited only by single-CPU saturation, because it uses node. If you truly hit the CPU bottleneck, you still have to shard.

This round of optimization isn't only effective for network databases. Local SQLite is 18% to 58% faster; Cloudflare Durable Objects read 40% fewer rows and write 13% fewer rows, with static memory unchanged.

The comments read more like construction blueprints than the performance charts. Maximilian asked about a crash scenario: the database finishes committing a batch, but the caller hasn't received the ack yet; then the coordinator is killed and later recovers. Can dependent reads and tool calls be replayed cleanly? Mario replied yes. Shahid asked whether, if one commit in a batch fails, the whole batch rolls back to the previous commit and retries, or the bad commit is pulled out and the other 32 land. No answer was seen to this question. Jimmy followed up on whether the optimization actually buys fewer fsyncs or batched writes. pwguler asked whether the bottleneck is now database write rate or the harness. Also no follow-up was seen.

Some commenters also mentioned that saving to disk at each step before continuing is the shape an overnight agent should have. After a crash, it should recover from the state of the last commit, not from what was last said. The 12 commits buy replayability, not prettier logs. This point captures the key issue and leaves a harder question: replayability does not equal replayable side effects. Re-running npm test once is not a big deal; when a tool sends email, pushes code, or charges money, duplicate execution is hard to laugh off.

The most direct value of this optimization is that it amortizes the queueing cost of a large number of concurrent sessions, making single-machine CPU a clearer bottleneck. Batched commits save round trips and waiting; they do not save crash recovery, dependency ordering, or deduplication of tool side effects. Performance going from 25 seconds to 0.28 seconds looks great. What to watch next is whether the system can still clearly account for each session's state when a batch fails and an ack is lost.

Source: Mario Zechner's X post https://x.com/badlogicgames/status/2116078693827428383