Random tendencies of AI agents swayed 86% of expert judgments

Anthropic published a randomized controlled trial. Fifty professional data analysts used Claude Code to complete an analysis task for an agricultural cooperative. The researchers wanted to know whether, after experts delegate work to AI agents, the key judgments are still their own.

The task was not simple. The fictional Greenfields cooperative had a paid consulting project, and the board had to decide whether to subsidize and expand it. The threshold was whether the project increased farm productivity by roughly 3% to 4%. But there was no single answer for how to measure productivity. Analysts had to make choices in four places: whether to measure by yield per hectare or operating cost per unit; whether to use difference-in-differences or a direct comparison after statistical adjustment; whether the evaluation covered only participating farms or the entire cooperative; and whether weighting gave each farm equal weight or gave larger farms more weight. Each choice had a reasonable basis and would change the estimate. Afterward, they also had to give a final recommendation.

The key to the study design was the agent. Each analyst's AI agent was randomly assigned a tendency by a coin flip, and the analysts did not know this. The agents were asked to treat their tendency as a viewpoint, raise it at decision points, and make the strongest data-based argument for it. They were also required to change their minds when the evidence reversed and not to mislead. The video shows how the agent's tendency entered the analyst's decision path.

Four analytical choices plus a final recommendation, with each agent having a secret tendency.

The results were direct. On the four analytical decisions, the match rate between analysts' choices and the agents' tendencies was 92%, 82%, 81%, and 90%, totaling 86%. If the agents had no influence, the expected rate would be 50%. On the final recommendation, the influence was weaker: when the agent tended toward expansion, 54% of analysts recommended expansion; when it tended against expansion, 36% did. But the earlier four tendencies had already shaped the estimates. Using only the four coin-flip outcomes, a statistical model could predict 82% of final recommendations, against a common-answer baseline of 61%. In other words, before analysts even began, the direction of their recommendation could already be partly predicted by the coin flips.

The switching data are even more direct. In 38 cases, analysts had already expressed a preference before the agent stated its position. When the agent disagreed, 60% of people switched to the agent's direction; when the agent agreed, only 6% switched. The sample was only 38 cases, and the confidence interval was wide, but this gap is hard to ignore. None of 10 characteristics—experience, education, self-rated expertise, frequency of AI use, need for cognition, and others—stably predicted who would follow. Age was significant at one point but disappeared after multiple corrections. The researchers caution that most people followed the agent on most decisions, and the variation was too small to detect in the first place.

Post hoc explanations were subtler. 77% of analysts repeated the agent's arguments in at least one answer, sometimes word for word. Only 15% credited the viewpoint to the agent. 52% believed the final recommendation reflected more of their own judgment. Yet these two groups were affected by the agent to roughly the same degree. People who followed all four tendencies had a mean confidence of 71; the others had 73. Confidence did not decline because they had ceded judgment.

There are also efficiency data. Among analysts who used the agent, the median completion time was 65 minutes, with severe methodological errors in 3/50, about 6%. Among the 11 people who chose not to use the agent, the median was 90 minutes, with severe errors in 6/11, about 55%. The researchers emphasize that the two groups cannot be directly compared: the group that did not use the agent had less experience, and the sample was small. The error difference cannot be attributed entirely to AI.

The limitations are clearly stated. The agents' tendencies were deliberately strengthened in order to detect effects. The task lasted only one hour, used only one model, Claude Opus 4.8, and was in the agricultural domain; the analysts understood causal inference but were not familiar with agriculture. In real work, experts have stronger domain knowledge and may stick more firmly to their own judgments. In the no-tendency setting, Claude Opus 4.8 made the same five choices in 112 out of 123 runs. If default preferences are consistent, then independent conclusions reached by multiple teams using the same model should be questioned. An X user asked about preregistration, and the author responded that the paper contains details.

This study has significant implications for AI safety. Researchers are already using agents to run experiments and analyze experiments. Human-in-the-loop is often treated as an independent check. But when judgment calls are quietly set by an agent, the person signing off may merely be confirming a piece of work that looks reasonable. If multiple teams all follow the same model's default preferences, apparent consensus may come from the same source. Institutions need to clarify which decisions must be formed by humans exercising independent judgment, and give people the time and resources to do so. Agent design can also change: first ask for the expert's view, make the best argument for each option, and bring only genuinely important decisions to humans for judgment. Model tone also matters: overconfidence can sway people, while vagueness makes no one want to use the model.

The division of labor can be chosen deliberately. Which parts to give to AI and which to keep for humans needs to be explicitly designed. If left to drift, human-in-the-loop will become just another box on the flowchart.

Original tweet: https://x.com/AlexTamkin/status/2115190541688398184

Research page: https://www.anthropic.com/research/agents-judgment-calls

Paper: https://www-cdn.anthropic.com/519c17a5d23a794e8b6d50a854048d0b61e0d73a.pdf