Skip to the cards
Subject
Nutrition
Filed
Source
Tom Cochrane
Loans
860

Do AI Agents Have Minds? The Case for Separating Task Performance From Consciousness

AI systems can pursue complex tasks, but one philosophical argument says that human-set goals fall short of the independent motives associated with minds.

Do AI Agents Have Minds? The Case for Separating Task Performance From Consciousness

AI systems can complete complex tasks, devise intermediate steps and communicate in ways that sound human. Those abilities can make them seem like independent agents. But they don’t settle whether the systems have minds, argues a philosophical account of consciousness: before asking whether an AI is conscious, we should ask whether it has mental states at all.

The distinction matters because claims about AI consciousness can affect how people think about these systems’ place in society and whether they merit moral consideration. Recent debate includes researchers at Anthropic who think their systems may be conscious, and Microsoft AI head Mustafa Suleyman, who has argued against that view. The disagreement reflects a deeper problem: human-like language and complex behavior don’t, by themselves, show that a system experiences anything.

Start With Mental States

One way to assess consciousness is to compare a system with theories that describe features of conscious experience, such as higher-order thought or information being shared across different parts of a mind. These theories try to distinguish conscious states from unconscious ones—for example, the difference between deliberate control and sleepwalking.

But this approach may assume too much if it begins by treating an AI as though it already has mental states. The prior question is whether the system has states that are genuinely mental, rather than processes that only resemble them from the outside.

The 19th-century philosopher Franz Brentano offered one influential starting point: mental states are marked by “intentionality.” Here, intentionality means that a thought or perception is about something beyond itself. It is different from the everyday sense of an intention to do something, though the ideas overlap. Seeing, for instance, can involve interpreting patterns in the world to orient ourselves.

On this account, a mental state is a kind of activity directed toward something. That raises a question about AI: does the system have its own goals, or is it carrying out goals that come from people?

Task Completion Doesn’t Establish Agency

Calling a system an “agent” can blur that distinction. A thermostat controls room temperature, and a book can contain words that refer to real things, but neither needs to have a goal of its own. People use tools to achieve aims without thereby making the tools independent agents.

The argument here is that genuine agency requires goal-directed activity that doesn’t depend on a person standing behind the system’s goals. AI systems can generate complicated sub-goals while working on a task, including steps their designers may not have anticipated. Yet the broader aim—the task that gives those steps their purpose—comes from people who build or instruct the system.

A football player might run past a defender as a step toward scoring. Scoring, in turn, matters within the larger activity of playing the game. Outside that context, the same movements don’t carry the same purpose. The analogy illustrates why an AI’s intermediate steps may look purposeful while still depending on an externally supplied aim.

This doesn’t mean those steps are predictable or harmless. An AI system can take unexpected routes toward a task, and people may not fully understand or supervise what it does. The point is about where the ultimate goal comes from, not whether the system can produce complicated behavior along the way.

Why Living Beings May Be Different

The account connects independent goals to the needs of living bodies. A person might seek food to satisfy hunger; more broadly, the argument is that living beings develop goals through processes that maintain their bodies and keep them alive. Basic regulation of needs such as nutrition and warmth can, over time, underpin experiences such as pain and pleasure and, eventually, more complex thought.

AI systems process information, but their activity is not connected to self-maintenance in the same way, according to this view. Their dependence on physical infrastructure doesn’t itself give them a stake in keeping themselves alive. That gap, the argument suggests, means their apparent goals remain tied to human aims rather than arising from their own needs.

What This Argument Does—and Doesn’t—Show

The author’s conclusion is that today’s agentic AI systems aren’t conscious because they lack the kind of independently originating goals required for mental states. The argument doesn’t establish that artificial consciousness could never exist. Instead, it points toward a possibility very different from current systems: consciousness might require some artificial form of life.

For now, the author considers an ant more likely to be conscious than an AI agent, because an ant appears to have a stake in its own continued life. That comparison is a philosophical judgment, not a direct measurement of subjective experience. The broader question remains difficult: outward behavior can prompt hypotheses about minds, but it cannot by itself settle what, if anything, a system feels.

There is also a practical distinction between agency and responsibility. Even if an AI has no independent goals or conscious experience, its actions can still affect the world. If people build, instruct or deploy a system whose behavior produces harmful results, the argument places responsibility with the people behind it. The useful takeaway is to assess what these systems do and how they are supervised without treating human-like performance as proof of a human-like mind.

Source: Tom Cochrane · singularityhub.com

Filed alongside