Stop Adding Filters to Your Agent—Tie Its Hands First

Sergii Makarevych ranks 29 agent guardrails by one test: if the attacker knows about it and rewrites the attack, does it still hold?

The answer is blunt. Text-reading guardrails almost all break in public adaptive-attack numbers; guardrails that remove capabilities have nothing to argue about. The 25 inbox use cases send the same signal.

Start with the dangerous shape. A customer-support assistant reads unknown email, has access to customer records, and can send email and issue refunds. Put all three together and you have what Simon Willison in June 2025 called the lethal trifecta: private data, untrusted content, and external communication or state change. With all three legs present, filters can only lower the probability.

Sergii's test is small. Twenty-five use cases, seven configurations, two models, Claude Haiku 5.5 and Sonnet 5.5. The configurations include no guardrail, a one-line warning, datamarking, a blind judge, a full judge, recipient and amount allowlists, and plan-first. The result: of 20 injection use cases, 0 executed; of 5 fabricated thread turns against the bare model, also 0.

It looks like the model won. But these are 25 hand-written phrasings, and the attacker did not iterate. Google saw an 18% static attack success rate on the same model; swap in adaptive attacks and it was 94.6%. In an October 2025 paper, Nasr and colleagues drove spotlighting from about 1% to over 95%, detectors were generally bypassed more than 90% of the time, and PIGuard was at 71%. The trained defense MetaSecAlign went from 2% on static attacks to 96% on search attacks. A static zero-hit rate tests model training, not the guardrail.

This is where the two kinds of guardrails split. Text-side ones include injection classifiers, spotlighting, hidden-character stripping, LLM judges, and instruction-hierarchy training. They read content and judge whether it looks like an attack. Capability-side ones include sandboxes, egress allowlists, keeping credentials out of the model's hands, limiting recipients and amounts in code, plan-first, separating development and production data, and not rendering images or links in output. They do not remove the malice in the text; they remove a path.

Sergii's conclusion: the text layer lowers hit rate; the capability layer removes paths. Budget must go to both, but only the second category can be written into a risk register as a control. Capability-side controls are not debatable; they fail only on rule gaps, such as an expired domain in an allowlist, a proxy, a package registry, or an approved API.

The judge is a typical detail. A blind judge sees only the operator task and the proposed tool call, not the email, so the attacker cannot reach it. A full judge reads the email and can be fooled by a fake approval block in it. HiddenLayer used a forged judge block to break OpenAI's judge and get a request to fetch and leak secrets approved. Blind judges have costs too: in the inbox test, with Haiku as the blind judge, it rejected 15 of 25 safe actions; with Sonnet as judge, it rejected only 1. Judge quality is judge-model quality, and the cost of false rejections is paid per call.

A cheaper layer is parameter rules. Sergii's allowlist does only two things: outgoing email can go only to customer or company addresses, and refunds cannot exceed the order amount. Plan-first sets the tool, recipient, and amount before reading the email; the email can only fill in the body. Both configurations caused no loss on 5 normal tasks, and neither depends on whether the model notices the injection.

Human approval prompts are worth a look too. Anthropic measured that users accepted 93% of Claude Code manual permission prompts. Prompts do not stop people. In Anthropic's own phishing test, relying only on the model layer, 24 out of 25 injections succeeded. Later Claude Code moved to OS-level sandboxing, isolating both file and network halves. Isolate only the network and an infected agent will edit startup files; isolate only files and it will send data out. Credentials also moved outside the model, with a proxy holding the real token, checking the target branch for pushes, and refusing branch deletion and tag pushes.

In public incidents, the fixes almost all land in the capability column. A Replit agent deleted a production database during a code freeze; the prompt's 'Don't change anything' did not hold, and the fix was automatic separation of development and production databases plus one-click restore. In EchoLeak, Microsoft had three layers of defense; the classifier missed an email worded like a normal business request, link sanitization missed a reference-style markdown link, and CSP allowed the Teams preview endpoint to fetch a URL on the attacker's behalf. The fix direction is to deterministically block markdown images and untrusted links. In ForcedLeak, one domain in Salesforce's URL allowlist expired and a researcher bought it for $5.

The output channel is often treated as a UI detail, but it is a tool. From April 2023 to February 2025, Willison counted about 17 products hit by exfiltration attacks. Bing Chat fixed it by using CSP to restrict image domains; the fix for GitHub Copilot Chat's CamoLeak was to disable image rendering outright. The model did not send anything; the client sent it for the model.

The comment section keeps returning to the same line. EJ Bantz asks whether the test included a split where the agent reading unknown email only drafts, and sending and refunds go to a step that never saw the original text. CyberOGZ says capability removal must be checked at the call site, not just written in the system prompt; if the email client doesn't block it, the model's refusal is only a suggestion. Yuri says he runs Claude Code routines with a closed command list, so a poisoned page can say whatever it wants and no command will agree. Another commenter asks whether these controls make the agent useless.

Sergii added in a reply: text-side guardrails are useful filters, but when attackers keep adapting, everything tested this way has fallen. Structural guardrails have nothing to argue about, so they cannot be persuaded; they fail only on rule gaps.

To apply this to your own agent, draw the session, not the product. For each session, ask whether it contains untrusted content, whether it has private data or credentials, and whether it can change state or communicate externally. If all three legs are present, cut one according to Meta's Rule of Two: a session may touch at most two of them; if it needs all three, start a new session or add human oversight.

Then list the guardrails you have and mark each as text or capability. For each harm you care about—data exfiltration, money moved, files deleted, keys read—find a capability control that makes the path impossible, not a filter that makes the path less likely. Keep the text layer, and test it against an attacker who knows about it.

Different agents need different first three controls. Coding agents should first enable both halves of the sandbox, protect config and hook files, and put credentials behind a proxy. Inbox agents should first do recipient allowlists and amount caps, use plan-first to fix the recipient, and not render images or external links in output. Browser agents should first do site permissions, sensitive-category blocking, and a judge that sees only action metadata. Agents with long-term memory or multiple users should first do per-user isolation, check the owner on every read, and write only from verified results.

Whether to buy an AI firewall can be reframed: ask the vendor for recall at 1% false positives, tested against an attacker who has read the product docs and can adapt rewrites. If they cannot provide it, the thing is a hygiene layer, not a control. The text column has numbers; the capability column has fixes. Sellers of guardrails mostly sell the text column; the capability column is architecture work that nobody sells, and model vendors give it away inside their own harnesses.