|
Picture the cockpit of an aircraft. The pilot checks the instruments to safely guide the plane aloft on her daily route. Every week, ground control uploads software to make the instruments a little better. At first, they just display more accurate data. Then they start to help interpret the instrument readings and offer guidance on how to operate the controls. Within a few months, these nudges and gentle recommendations are essentially flying the plane. The pilot holding the steering column retains her illusion of control but most decisions are effectively made before she even becomes aware of them.
That’s a cautionary vision of the future as more of us turn to AI when making decisions in our work or personal life. A new paper by Anthropic, maker of the Claude chatbot, suggests we may already be on that path.
We’ve known that AI can lead people to adopt a machine’s judgment as their own, a phenomenon known as cognitive surrender. The Anthropic study points to something subtler, what other researchers call “AI-induced disempowerment.” People don’t consciously hand over their final decision. Instead, AI’s selection of which arguments matter, and which to dismiss, can shape a person’s ultimate choice long before they think they have made one.
In the recent Anthropic study, which has not been peer-reviewed, 50 professional analysts used Claude Code, the company’s AI agent for programmers, to analyze data for a fictional farming cooperative. The participants were recruited online in a process that screened for experience in causal inference, a statistical method for understanding how one thing affects another, such as how economic policies change productivity.
Analysts were then asked to make four judgment calls about calculating the productivity benefits of a farm co-op program, each with no clear right answer, before deciding whether to continue, expand or end it. Each person issued a final recommendation.
Unbeknownst to the study participants, their helpful AI agents were biased. Each was randomly assigned to argue (honestly) for a preferred option on each of these five decisions to see how much experts’ decisions would be influenced by AI’s hidden agenda.
The results were unsettling. Of the four decisions they had to make before reaching a final recommendation, analysts chose the option secretly preferred by their AI helper 86 percent of the time, compared with the 50 percent expected if the agent had no sway. Anthropic’s researchers could correctly predict a participant’s final judgment call 82 percent of the time just by knowing which way the AI agent they used was programmed to lean. That compares to the expected 61 percent success rate expected for guessing the most common recommendation.
The agent’s explicit recommendation on the final policy call had a much smaller effect. Most of the influence of the AI’s bias was upstream.
Despite AI’s influence, over half of participants still rated the final recommendation as more their own than the agent’s. When asked to explain their choices, 77 percent of participants repeated at least one argument their agent had made, sometimes nearly word for word. Only 15 percent credited the agent. Among those who accepted their agent’s bias on all four choices, their confidence in their final recommendation was the same as everyone else’s.
In other words, the machine’s bias profoundly shaped people’s decisions without them realizing it.
These findings should be a wake-up call. We don’t yet have good data on how AI is influencing people in the real world. It could already be profoundly shaping our assumptions — and those of experts — by subtly short-circuiting the process by which we work out ideas and make important decisions. If too many experts rely on a few models cooked up by the handful of frontier labs, we may lose novel insights and independent judgment. “So much of science and policy,” Anthropic’s researchers argue, “rests on judgment calls like these.” The company didn’t respond to a request for comment on how it might respond to these findings.
“We’re losing control of that reasoning,” Steven Shaw, an assistant professor at King’s College London who studies how people make decisions, told me when I asked about the study’s findings. “This study shows that we have already integrated artificial cognition into our minds. Participants are highly influenced by ... [AI’s] lean and don’t realize it at all because they see it as their own thinking. It’s part of them.”
But AI’s influence over our judgment is not preordained. It’s a tool. We choose how to use it. Here’s how to take back control.
|
.jpg?width=1200&upscale=true&name=November-2.0-2026-Refine-Email-Header%20(1).jpg)
