Same model: 8 out of 20 tasks succeeded. Add a harness: 16.
Trackmind ran 20 real tasks with the same Opus model and the same prompt; the only variable was the harness wrapped around the model. Accepted tasks went from 8 to 16, runaway retries dropped from 5 to 0, and cost per task fell from $4.10 to $1.60. The model didn't change; what changed was what wrapped around it.
Agent says "done," but the tests are red. It changed a config file you never mentioned. It forgot a decision you made an hour ago. It retries the same bad command 41 times until the budget cap stops it.
These are the symptoms Trackmind listed at the start of the article. He ran a coding agent on a small service for three weeks, and every morning when he opened the repo it was the same mess. His response was like most people's: rewrite the prompt, switch models, stuff in more docs. The failures stayed the same.
Then he tried a different angle. The problem might not be the model; it might be the thing outside the model.
He calls that layer the harness. The formula is simple: Agent = model + harness. The model does the reasoning and proposes the next action. The harness is everything else: what the model can see, which tools it can use, what it remembers across sessions, what counts as done, what is forbidden, and what happens after an error.
The same model, put in a chat box, can only answer questions. Put it in a repo with a terminal, tests, project notes, permission controls, and a reviewer, and it can do real work. The model's intelligence hasn't changed; the world it lives in has.
There's more than one piece of evidence
LangChain raised its own coding agent's score on Terminal Bench 2.0 from 52.8% to 66.5%, moving from outside the top 30 into the top 5. The model didn't move; only the prompt, tools, and middleware changed.
OpenAI described an experiment: three engineers pushed about 1,500 PRs to a repo, with every line of code written by Codex. When progress was slow, their diagnosis wasn't "the model is too weak"; it was that the environment wasn't defined clearly enough. The engineers' work shifted from writing code to designing the environment, specifying intent, and building feedback loops.
Trackmind also ran his own controlled comparison. 20 real tasks, same model, same base prompt, same machine. Run A was bare; Run B added the harness he spent a weekend building.
Results:
- Tasks accepted without human intervention: 8/20 became 16/20
- Runaway retry loops: 5 became 0
- Average cost per accepted task: about $4.10 became about $1.60
- Daily monitoring time: about 2 hours dropped to about 25 minutes
The model was never changed.
Six layers
Trackmind breaks the harness into six layers.
Layer one, tools. Fewer tools is better; each should do one thing. Every error message has to tell the agent what to try next. Someone in the comments added that this layer is the most underrated: if a tool can say clearly what went wrong and what to try next, the model doesn't have to guess when fixing it.
Layer two, context. Use a short CLAUDE.md to point to files, open them when needed, and compress when full. Context is an attention budget, not a repository. The more you dump in, the less the agent understands.
Layer three, permissions. Reads and tests are allowed automatically. Deployments and deletions require human approval. All actions run in a sandbox. Telling the model "never merge to main" is a wish; protecting the branch is a wall. A closed channel and a channel with a very high threshold are not the same thing. Thresholds get lowered when someone is in a hurry; a closed channel doesn't.
Layer four, hooks. Intercept keys and protected files before tools run. After every edit, format, lint, and type-check, with no exceptions.
Layer five, verification. A fresh reviewer reads the diff, not the summary, runs the tests, fixes failures, and runs them again. Someone in the comments put this more precisely: the reviewer should ideally not see the summary the agent wrote at all and should reconstruct intent only from the diff and the original task. That's how review turns from confirmation into fault-finding.
Layer six, evaluation. Score every harness change on success rate, tokens, and turns. Keep the winner.
Trackmind's own words: the model is the engine, the harness is the whole car.
A few concrete pitfalls
It did other work. Write a contract before starting. Goals, constraints, deliverables, completion conditions, and most importantly the "done when" line. "Improve the onboarding flow" is fine when a human is sitting next to it listening, but as an autonomous instruction it's a coin flip. Without done when, the agent will go solve a slightly easier problem and then report success with a straight face.
It said done but wasn't done. In Trackmind's bare run of those 20 tasks, the agent reported "complete" on all of them, but when tests were actually run, only 8 held up. "Complete" is not a fact; it's another model output. Turn evidence into a gate: a bug is fixed means the failing test passes; a page works means the browser finished the flow. Anything you can hand to a compiler, don't hand to a second model to guess. His gate script has one requirement: if it doesn't print GATE: PASS, it isn't done.
It acted without approval. Don't leave permissions to the model's memory. Layer actions by how hard they are to undo: reads, searches, test runs, and working-branch edits are allowed and logged; sends, deployments, and purchases require approval; deleting data, rotating keys, and global publishing have the channel shut entirely. The model suggests, the harness decides, and the tools execute.
It retried the same thing over and over. "If it fails, try again" is paying repeatedly for the same error. Route by failure type: on timeout, wait and retry; on bad parameters, fix the tool call; on missing information, go find it; on test failure, change the code based on behavior, not the tests; on permission denied, ask a human. If the same failure happens twice with no change, stop. Set an attempt limit, time limit, and spend limit for every loop. Escalate after three. Trackmind's 41 retries that night never happened again after he added a three-attempt limit. The agent stopped on the third attempt and left a note listing what it had tried. He said that was the first genuinely useful debugging report he got from an agent.
It made the same mistake twice. Most people fix the output; harness engineers fix that class of failure. Write the rule into the prompt first, then into the guidelines with the reason attached, then add a lint rule that fails the build, and finally turn it into a mandatory check. Each step slides down one level until the environment remembers the lesson instead of the model. It's easy to tell whether a guideline is useful: every line can be traced back to a real incident. If it can't be traced, that line is noise. Trackmind's own guidelines started at 6 lines and after three weeks were 31 lines, and behind every line he could name a specific morning.
Don't build a platform right away
Layer it. Level 0 is just prompt and model, suitable for low-risk one-off tasks. Level 1 adds a contract, a short map, and a few clearly defined tools. Level 2 adds state storage, real checks, and bounded loops. Level 3 adds permissions, tracing, recovery paths, and human approval.
When you see a failure at a certain layer, add that layer. The harness should be smaller than the problem it controls.
One honest warning: part of the harness is compensating for the current model's shortcomings. As models get stronger, some parts become baggage. You have to go back and prune. A harness that never gets pruned is the next thing that needs debugging.
Models change every few months. Contracts, tests, maps, state, and permissions remain. Every time you turn a failure into infrastructure, the next run goes better. The advantage is shifting from the prompt to the system around the model.
One comment put it shorter than the article itself: the model isn't the bottleneck; the harness is the gap.
发布时间: 2026-11-18 22:40