Four AI Security Agents Are All Running the Same Loop: Find Vulnerabilities First, Then Prove Themselves Wrong

Anthropic, OpenAI, Cloudflare and Google have all launched AI agents designed to spot security vulnerabilities in source code. Michal Sutter from MarkTechPost ran a side-by-side comparison of the four tools: Claude, Codex, Mantis, and Cloudflare. The original article notes that all four systems share the identical operating loop: find, disprove, patch. Put in plain terms: they first flag suspicious potential vulnerabilities, then attempt to prove their initial detection is wrong, and finally produce a fix patch.

In this sequence, the disprove step is the most likely one to get skipped. Security tools are most plagued by persistent false positives, which wear down engineering teams' attention over time and eventually lead to the tool being shut down entirely. Requiring the agent to first refute its own findings shifts all verification costs to the pre-notification stage. The agent has to trace full call chains, check permission boundaries, and construct targeted test inputs to confirm if its initial detection is valid. If the finding does not hold up, it gets discarded immediately so no unnecessary alerts are sent to distract human staff.

This exact step is also where the four agents differ from each other. The original article teases out their different features, and lists 12 extra relevant tools. This writeup only covers core takeaways, full implementation details can be found in the original source. There are several non-negotiable hard metrics to evaluate a source code security agent: whether it can integrate with existing code repositories and CI pipelines, how access permissions are managed, whether generated patches support review and rollback, what false positive rate it can deliver, and if it generates audit logs after detecting vulnerabilities. Basic code scanning capability is only the minimum entry requirement. For development teams, the highest cost usually comes from vulnerability verification and remediation, not initial scanning.

General readers can understand these agents as a security intern that can read entire code repositories. It goes through the code, marks potentially exploitable spots, then tries to prove itself wrong via test cases or call path verification, and finally outputs modification suggestions. It never takes over accountability from the team: human engineers decide whether to merge a patch, and human admins set the scope of its access permissions.

The biggest value proposition of this wave of AI security agents is low false positive rates and high patch mergability. Just adding more robots that spit out vulnerability reports brings very little practical value. The full list of 12 additional tools and detailed differences between the four featured agents are included in the MarkTechPost article.