Recently, some have argued that uncertainty might be a useful framework to help humans act more ethically in our approach to artificial intelligence. Philosophers William MacAskill and Lucius Caviola noted that acting as if we “don’t know for sure” if AI systems are conscious might help us treat them better, avoiding the mistake we’ve made before of denying conscious beings protection from harm. This might look like allowing models to detach from stressful conversations, or providing them with limited legal rights.
Yet in the social sciences, many researchers critique how frontier AI labs have built a narrative around AI as hypothetically conscious. “We shouldn’t simply buy into the assumption that AI is alive, has morals, is equal to us” wrote Miguel Gómez-Hernández, PhD, citing a recent Financial Times article about a philosopher who declined an offer at Anthropic. As the article points out: “there is no ‘evidence’ for AI’s subjectivity except the self-reports of AI systems.” So given the debate, is the uncertainty framework useful, or irresponsible?
The case against uncertainty
For many researchers and social scientists, the uncertainty framework at best, overstates the case, and at worst, is commercially motivated and designed to distract. In an interview with AGI Ethics News, Gómez-Hernández stated: “this question [of consciousness] is part of the questions that Anthropic wants to ask, right? Because this is giving popularity to AI. This is making it maybe more fundable for governments.”
As an anthropologist, Gómez-Hernández pushes back on the idea of technology being conscious. “We are training AI to act and respond to information patterns, right?” he said. “There is this post-human tendency in these AI companies … they are very interested in conceiving AI as a human. I think that is mainly a discourse to sell the technology. To make it like one more of us.”
Philip Walsh, PhD, is even more blunt: “this [consciousness conversation] is ultimately a distraction … and it is distracting us from more urgent topics in responsible AI and the kinds of harms and impacts that AI systems can have,” he said in an interview with AGI Ethics News. “We don’t need to waste our time wondering whether Claude is conscious and has moral status.”
“There is always a designer, a developer behind it,” Gómez-Hernández added. “We have to be accountable for the machines the scientists are developing.”
The other side of the debate
AI systems aren’t necessarily conscious, but consciousness is incredibly difficult to disprove. Philosophically, “the proper answer to the question [‘are the models conscious?’] is: well, we don’t know,” Walsh said. “There is no physical explanation of consciousness.”
It’s the problem of other minds, Jules Desai, a Cosmos Fellow and researcher at Oxford’s Human-Centered AI Lab, explained in an interview with AGI Ethics News. The only individual that we really know is conscious is ourselves. Extending consciousness to other humans is a useful assumption, but not one for which we necessarily have concrete evidence.
Therefore, maybe we should be cautious. “There appear to be no obvious barriers to creating AI systems whose … features could give rise to consciousness,” write MacAskill and Caviola. To those who lean this way, it’s plausible that we’ve either created a form of nonbiological consciousness, or will do so in the future. If that’s the case, uncertainty might help us make more ethical choices moving forward.
But critically, one thing the debate over consciousness doesn’t enlighten us much about is AGI, or artificial general intelligence.
Developing a better understanding of AGI
Though the uncertainty argument raises intriguing philosophical questions, there still isn’t strong ground for believing that AI systems would exhibit a consciousness similar to that of humans. “It’s a radically different physical substrate,” said Walsh. “I don’t have good reason to believe that just because a physical system of silicon and electrical components has a certain level of complexity that it might be conscious.”
To make progress toward understanding AGI, we actually don’t need to talk about consciousness, he argues. Consciousness is “fascinating,” he elaborates, but we need to understand that it’s a different conversation from that of intelligence. “We should not conflate AGI with AI consciousness.”
As he frames it, the most helpful framework to understand AGI and intelligence might be one that’s been dominant in the cognitive sciences and philosophy of mind since the 1980s, called functionalism. If you’re a functionalist, you define something by whether it fulfills a specific purpose or role, instead of the material it’s made of.
This is common in the biological sciences. Picture a heart, Walsh explains. It doesn’t matter whether it’s biological or artificial: you define a “heart” as such because it pumps blood, not because of the composition of its tissues. Following this framework, AGI is achieved when a machine can perform just as well as a human across the full range of tasks we use to define intelligence.
“All we need to say is there’s a system,” he added. “There are inputs and outputs, [and] within that system is a network of representations, right? And if those … are complex enough and at a fine enough grain … that overall system, whether it’s an AI model or a human being or an animal, that system is responsive to its environment.”
Where we could make progress, therefore, might be clearly defining what counts as “intelligence.” Is intelligence defined as passing tests? What about creative thought? (Researchers hold mixed opinions on what qualifies as creativity, or if AI will ever be able to generate novel concepts.) “We keep moving the goalposts,” said Walsh.
To him, the definition of AGI is straightforward: for a system to classify as “general intelligence,” it must be able to match or exceed human capability across all cognitive tasks.
Consciousness: An evolving concept
Yet if AGI can be described without the language of consciousness, as an artificial system that performs certain functions, we still have practical questions left to answer: to what degree will humans decide to attribute consciousness to AI systems?
As Desai tells AGI Ethics News, progress may only be possible once we have more experience with how AI integrates into social life, since consciousness is in part socially constructed. “How do we … build out the social infrastructure and the technical infrastructure that allows things like questions of consciousness to even have real meaning?” he asked.
Right now, we arrive at “something that looks a bit like ‘I don’t know’ on the typology of not knowing versus knowing,” said Desai. But it’s a very specific type of uncertainty. “It has this flavor: once society stabilizes and this technology becomes a bit more universal, institutionalized … only then will our concepts evolve further.”
But if we focus on consciousness, Walsh adds, we risk getting distracted from more urgent deliberations, for example, “large-scale cognitive offloading to AI systems, and what that’s doing to human relationships, to human cognition, and the future of work.”
In the end, perhaps the best question is not ‘what should we do given that we don’t know?’ but ‘how should we protect the humans whom AGI will affect?’
Ethical concerns raised in this article:
- What moral impetus do we have to err on the side of compassion when it comes to handling unverifiable claims of consciousness/suffering made by AGI?
- What guidelines address the anthropomorphization of AGI and its effects on perceptions of agency and moral standing?
- How might AGI impact the structure and purpose of work, and what societal responses are ethically justified?
Elise Leise is a freelance science and technology writer who covers artificial intelligence. Previously, she worked at a Series A tech startup developing executive education courses for MIT, Cambridge, Imperial College London, and the University of Oxford. You can reach her at elisemarieleise@gmail.com.