Wink Pings

When AI Reviews AI's Replies, the Biggest Fear Is That It Becomes a Rubber Stamp

Ajitesh Shukla built a procure-to-pay agent with LangChain, splitting drafting and approval across two models. The mechanics are rigorous, but commenters have already run into the pitfall of a reviewer being led astray by context.

Letting an AI agent grade its own homework looks efficient in the short term, but in the long term it is an audit disaster.

Ajitesh Shukla shared on X a procure-to-pay (P2P) agent built with LangChain: drafting replies is handled by one agent, and approval by a separate reviewer subagent. The design does not have the drafter also self-check; instead, review is split out.

Its mechanics are very concrete:

  • the reviewer runs on a different model and cannot see the drafting agent's reasoning process;
  • the system force-calls the reviewer every time; review cannot be skipped or explained away;
  • the reviewer's conclusions are logged separately for auditability;
  • when adding new business categories, new review skills are added while the drafting agent remains unchanged.

image

This separation addresses an old problem in approval chains: drafters tend to rationalize things for themselves. Using a different model, isolating reasoning, and forcing invocation are all meant to keep review from being carried along by the drafting process. Some commenters endorse this, saying that a different model plus not seeing the reasoning can keep reviewers honest; others mention that independent-model review can create a clean audit trail and help control bias.

But what is more worth reading in the comments is the risk of failure. Jess asked whether the reviewer would start rubber-stamping or would actually push back. Timilla offered more concrete experience: his reviewer subagent completely collapsed as soon as it read the main thread; after stripping out the trace, the rejection rate soared. He also asked how the original ledger row should be formatted.

This shows that the key to independent review is not the name but the input boundary. If the reviewer sees too much drafting context, it is easily persuaded and ends up as a rubber stamp; only by giving it the necessary facts and clear criteria can it really catch mistakes. Forced invocation and separate logs are the baseline; information isolation determines review quality.

For people building agent workflows, this case can be used directly as a checklist: who drafts, who approves, what the approver can see, whether invocation can be skipped, and where conclusions are recorded. The same applies to ordinary users: when AI handles processes such as procurement, refunds, and customer service replies, the biggest fear is not that it writes poorly, but that no one truly reviews it.

Original post: Ajitesh Shukla

发布时间: 2026-11-16 00:25