Breaking Down Claude’s Memory System: 413 Memory Files Lose to 9 Folders

The Anthropic engineering blog published a teardown of agent memory. The title is serious, the case study is brutal: in Pokemon Red, Claude Sonnet 3.7 wrote 413 memory files, stuffed into 225 folders, and earned 1 badge; Fable 5 wrote only 9 files, plus one global.md, and earned 4 badges.

The blog defines agent memory as knowledge Claude acquires from experience and can retain across sessions. It does not change model weights, nor is it the same as in-context learning within a single conversation. In implementation, it looks more like a layered file system: at the bottom are transcripts, searchable raw event logs; above that are memory notes Claude writes itself; above that are memory representations such as a wiki; and at the top is injected context preloaded before each session begins.

The middle stage called dreaming is, plainly put, offline tidying. When idle, the model reviews logs, corrects errors, writes notes, builds indexes, and pushes content that should be preloaded to the top layer. Some commenters called dreaming a fancy name for garbage collection, while others said it is just organizing Markdown files with a few extra steps. Crude phrasing, but not far off.

The Pokemon comparison makes the point best. Sonnet 3.7 assumed by default that everything might be useful, recording every battle, every item, and every NPC separately; as a result, the game state kept changing, and many notes expired almost as soon as they were written. Around step 10,000, it started building a fourth set of master memory files; over the next 17 steps it created 6 files and deleted 9, without taking a single in-game action.

Fable 5 was the opposite. Its global.md acted like a dashboard, holding the current goal, the plan for gym battles, and memory rules it had summarized itself. locations held route plans, and gameplay held validated strategies. For example, after failing to catch a Magikarp with seven Poké Balls in a row, it wrote: if more than three balls fail in a row, change the rhythm and insert extra waits between menu steps. The next time, it actually caught it.

In the CL-Bench database task, the gains from memory were direct. Opus 5.5 went from 0.655 to 0.838, Sonnet 5.5 from 0.644 to 0.833, Opus 5 from 0.564 to 0.828, and Opus 4.7 from 0.383 to 0.700. But imposing hard rules on the model backfired. After setting it to record only failures, Opus 5’s memory gain dropped by 0.234, and Opus 4.7’s dropped by 0.150. For strong models, judging for themselves what is worth remembering is more effective than hard-coding rules in advance.

Dreaming also has a concrete error-correction case. In Pokemon, Claude mistakenly thought the PokéCenter was on the east side, declared Mt Moon DONE as soon as it walked out the door, and then fell back down the original path from a one-way ledge. The navigation tool did not show the fall, so it instead concluded it had overshot and kept going in circles in the wrong direction. After having dreaming check that note against the transcript, the error was fixed; Claude re-entered the cave and eventually reached Cerulean City. The original memory version, after falling off the ledge three times, wrote: I’m going to stop rather than keep looping.

The wiki is another layer. Anthropic tested with about 350,000 files and roughly 420 million tokens of Stainless internal documentation, having an Opus 5 agent continuously build a wiki, edit pages, merge duplicates, and verify sources across 128 sessions. Its score on 25 questions rose from 13% to 48%. The combination of wiki plus search tools rose from 54% to 61%, while search tools alone were at 53%. Search plus wiki also saved about 20,000 tokens per question on average.

But a wiki is not a cure-all. In Harvey LAB’s M&A legal briefing task, more than half of the 24 questions required finding information across multiple documents, and the wiki raised the score from 73% to 87%. In another type of task, where most criteria could be found in a single document, the wiki showed no clear gain. More awkwardly, in one case the wiki grew larger than the source material: 10 documents totaling 115KB became a 68-page wiki with 255 links and 203KB, 1.77 times the original. The blog invoked Borges’s story of a map the size of the empire; the point is clear: abstraction failed.

Multi-agent collaboration also has pitfalls. In a contract-clause task, several agents wrote simultaneously to a shared provisions.md, overwriting one another; in the end, only 41 of 57 scored facts remained, and the briefing score was 72%. After switching to a separate page per contract, with one agent quoting clauses verbatim, all 57 were preserved and the score was 100%.

Injecting context at the top layer requires even more restraint. Claude Code’s auto memory does not stuff all memories into every session; it loads only the first 200 lines or 25KB of MEMORY.md, treating it as an index. The blog warns that injected memories consume tokens, erroneous notes get carried into every session, and the longer the instruction file, the more adherence may decline.

When recommending the blog, elvis said that no one-size-fits-all memory system exists today. A commenter added: the most advanced memory system ultimately loops back to a few sensibly named folders. Gaetan Semet noted that for dreaming to be truly useful, it needs a trust hierarchy: findings at the bottom, supported facts above, principles at the very top, and the agent consuming only the trusted layers.

These discussions point to the same trouble: when to write memory, how to write it, how it expires, and how to handle conflicts between old and new facts still have no standard answers. As models get stronger, memory systems will keep changing. For now, what can be done is to first clarify folder naming, indexes, and expiration rules. Architecture diagrams can be drawn later.