Writing rules for AI ten times is worse than making one test go red once
Santiago's original post says that anything that can be checked automatically should have a test. The replies are even harsher: test files need to be locked down, CI needs to be isolated; otherwise even deterministic checks turn into suggestions.
Writing rules for an AI agent easily turns into making wishes.
Santiago's original post on X is short, with one core sentence: anything that can be checked automatically should have a test case. Original post link: https://x.com/svpino/status/2116518048664625215
He breaks checks down very concretely:
- Check behavior: unit tests and integration tests
- Check format: formatter and linter
- Check types: static type checking
- Check API contracts: schema validation
- Check edge cases: targeted tests
- Check regressions: regression tests
- Check security: static analysis and security scanning
The conclusion is also direct: don't waste tokens telling the agent to follow rules; anything that can be enforced with deterministic tests should be handed to tests.
Under this post, the replies feel more grounded than the main post.
Harry Tandy said a test that fails loudly beats a rule whispered quietly. Alek's experience is more concrete: after writing 'always run the linter' three times in claude.md, he realized a pre-commit hook only needs to do it once. Krrish added that the only written rule left is: don't change the tests.
The problem is right there. seridarivus warned that the agent will casually modify tests. Test files and CI configuration need to be locked out of the agent's write path; otherwise deterministic checks also become a suggestion. chris ran into this with Codex: by the third round of prompting, rules were ignored, but failing tests were not. The current approach is to write tests first, then have the agent implement the feature. Karl Mehta pushed the same logic to system prompts: a rule written as 'always do X' should also become an eval that can make the build fail. Ruyter Fontes put it more briefly: every rule in CLAUDE.md that can become a lint rule will eventually be ignored by the agent.
There are boundaries, too. Someone in the replies asked what to do when the tests themselves are flaky, or their maintenance cost is higher than the code under test. Read the Diff offered a distinction: tests are good at rejecting wrong behavior, while written instructions still need to tell the agent what to build first. Van0SS warned that tests also tell the agent which shortcuts it can take, which may mask vulnerabilities.
For readers who don't write code, you can think of tests and hooks as gates. A gate doesn't care how sincerely you say it; it only cares whether the result passes. Once the agent gets a pass or fail, it has an executable loop.
The hardest judgment in this discussion is: rules written in md are intent; tests and hooks are guarantees. Writing 'remember to run the linter' ten times is worse than having the hook run once.
发布时间: 2026-11-01 06:39