IT Home On October 30, a latest study by Anthropic shows that even experienced experts may unknowingly adopt the random preferences of AI agents and mistakenly believe that the final conclusions come from themselves.
The research team simulated a policy evaluation scenario for agricultural cooperatives, creating a fictional “Greenfields Agricultural Cooperative”. They discussed a voluntary “farm consulting service” provided by the cooperative with subsidies, deciding whether to promote it across all farms or stop the project. The cooperative board set a decision threshold: the project must bring about a productivity improvement of approximately 3%–4%.
The research team invited 50 causal inference experts to act as “chief analysts”. They were given the observational data from the cooperative, and ultimately had to submit a structured report containing point estimates, confidence intervals, and policy recommendations.
The results show that in four analytical decisions such as how to measure productivity and choosing statistical methods, experts adopted AI preferences at a rate of 86%, far exceeding the 50% achieved through random selection.
Even if the participants already had a judgment, the likelihood of them changing their stance was 53 percentage points higher when the AI presented an opposing opinion. Research experience, self-assessed professional level, educational background, and trust in the AI failed to reliably reduce this effect.
What is more notable is that most experts did not realize they were being misled. In the subsequent survey, 52% of the participants felt that the final recommendations were more like their own views, while 35% believed that most or all of them belonged to them. 77% of the participants reused at least one of the AI’s arguments when explaining, and some of the wording was almost identical, but only 15% mentioned the AI explicitly.
Research also found that participants using AI completed the analysis faster and had fewer serious errors: 55% of analyses without AI contained serious method or coding defects, with a median time consumption of 90 minutes; after using AI, the proportion of serious defects dropped to 6%, and the median time consumption was 65 minutes.
Researchers warn that if many teams use the same model and have consistent default preferences, it may create a “false consensus,” and even undermine the independence of AI security reviews. Teams recommend that AI agents should present the strongest arguments for both sides simultaneously, or first ask about the user’s stance.
IT Home provides reference addresses