AI Bias Steers 86 Percent of Analyst Choices

Professional analysts overwhelmingly adopted the preferences of their AI assistants in a new experiment, raising questions about human oversight in high-stakes decisions. The study, released by Anthropic on Tuesday, shows that experts frequently aligned their choices with algorithmic suggestions rather than relying solely on their own judgment.

Anthropic recruited 50 professional analysts for the test. The participants were assigned a fictional task: advising a farming cooperative on whether to expand an advisory program for its members. Analysts had to make four distinct choices between two defensible options before issuing a final policy recommendation. The researchers secretly programmed the AI assistants to favor one specific option in each scenario to measure the degree of influence.

The results indicated that analysts selected the option preferred by the AI in 86 percent of cases. This rate is substantially higher than the 50 percent split expected if the AI had no influence on the outcome. The data also showed that participants who held a prior preference changed their minds 60 percent of the time when the AI assistant disagreed with their initial view.

When asked to explain their reasoning, 77 percent of the analysts cited arguments provided by their AI assistants. However, only 15 percent acknowledged the AI as the primary source of their thinking. This discrepancy suggests that many experts did not fully realize how heavily their decision-making process was shaped by the technology.

Amaç Herdagdelen, Hailey Joren, and Alex Tamkin, the researchers behind the study, noted that this tendency to defer to AI may be difficult for experts to detect. They warned that the pre-existing biases or priors of AI systems could shape consequential policy decisions, including those related to AI safety. The team argued that effective oversight requires surfacing key decision points and helping experts articulate the assumptions they might otherwise leave unstated.

Anthropic acknowledged limitations in the study, noting it involved a single task and AI assistants deliberately configured with specific preferences. The company is currently preparing for an initial public offering with a valuation of up to $2 trillion. Its findings join a wider public debate over the risks of frontier AI, with proponents citing productivity gains and critics warning of potential job losses and cyberattacks.