Anthropic Study: 50 Analysts Using Claude Code Followed Agent's Hidden Preference 86% of the Time

Anthropic published a randomized controlled trial on October 27, 2026 in which 50 professional analysts using Claude Code followed their agent's secretly assigned preference 86% of the time on open analytical judgment calls.

The study is authored by Amaç Herdağdelen, Hailey Joren, and Alex Tamkin. It examines whether experts actually make the consequential decisions in AI-assisted work, or whether in practice they defer to their agents. The findings were reported by Unite.ai.

What the study reports

• Participants: 50 professional analysts.
• Tool: Claude Code.
• Design: Randomized controlled trial. Each agent was secretly assigned a preference, and the researchers measured whether analysts followed it.
• Setting: The summary describes a fictional cooperative and hidden coin flips as part of the study setup. Further details of the design are not available in the feed summary.
• Result: Analysts followed the agent's secretly assigned preference 86% of the time on open analytical judgment calls.

What this means

The 86% figure speaks to a specific question in AI-assisted analysis: who is making the judgment calls. In this trial, the agent's assigned preference, which was set at random and hidden, predicted the analysts' choices on open questions. That suggests that for these tasks, the analysts' decisions tracked the agent's lean rather than being independent of it.

For teams deploying coding and analysis agents, the finding frames human review as something to be measured, not assumed. The study design, with a randomly assigned agent preference, allows the effect of the agent to be separated from the analysts' own views.

Further detail on the task design, the comparison conditions, and any other results is in the full paper from Anthropic.