One of this publication’s primary goals is to capture the zeitgeist surrounding the presumed-imminent emergence of AGI. It’s important for us to discuss the news and analyze research papers, but we intend to take our exploration of AGI ethics beyond the headlines and preprints.
There’s an entire world of conversation and debate unfolding across a multimedium spectrum of content platforms and, often, the most meaningful discourse can only be found by those who know where to look.
I want to take you on a journey into one such trove to illuminate a fascinating debate on the topic of AI consciousness. Not only does it capture the general tenor surrounding AGI ethics as of the time of this newsletter’s publication in September of 2025, it also illuminates the deeper challenges associated with defining “identity and self” in the context of AGI.
A manifesto, a moment
Our story begins with the publication of a preprint research paper entitled “AI Consciousness: A Centrist Manifesto” by Jonathan Birch, a professor in the London School of Economic’s department of philosophy, logic and scientific method.
Per the paper, humans face “two urgent challenges concerning consciousness and AI.” The first, is that people are likely to confuse AI that pretends to be conscious with AI that is. And the second is that, as Birch puts it, “profoundly alien forms of consciousness might genuinely be achieved in AI, but our theoretical understanding of consciousness is too immature to provide confident answers one way or the other.”
Professor Birch’s paper “attempts to construct mutually consistent strategies for addressing both challenges.”
Birch is a well-respected figure in the field of AI consciousness and his recent work includes co-authoring papers alongside world-renowned researchers such as Jeff Sebo, Yoshua Bengio, Geoff Keeling, and Eric Schwitzgebel.
His paper prompted a whirlwind of discourse with reactionaries taking to their blogs, social media accounts, and community forums to chew on Birch’s ideas, test their merit, and share insights with each other. While it’s clear that the “manifesto” had an impact on people interested in the discussion, it’s also refreshing that none of that discourse appears to have devolved into the negative exchanges that sometimes emerge during this sort of science-based, online debate.
The greater good
Professor Birch commented on his reasoning for writing and publishing the “manifesto” paper in an August 28 post on the Effective Altruism Forum.
“I wrote it because I realized that I’d co-authored a string of pieces and yet none of these quite captured my personal position as of 2025 … I’ve seen over the past year extreme positions on both sides becoming entrenched … I think the best approach goes calmly down the middle, acknowledging reasonable points [on] both sides.”
A user on the EA forum with the handle “alx” responded that “there is little evidence either way” for the potential for radically un-humanlike forms of consciousness to exist, adding that both sides’ views are “equally feasible.”
Meanwhile, over on X, Birch received some friendly criticism from AI alignment researcher Aaron Bergman. They responded to a post from Birch sharing the paper with a critique focused on the manifesto’s perceived weakness in describing harm vectors for the two identified challenges.
As Bergman put it, “at the end of the day I think this cashes out in utilitarian terms. There ought to be some stronger theory of harm IMO.”
Reasonable Debate
A few weeks later, in Mid-September, Birch’s manifesto became the subject of several blog posts. Over on Substack, philosopher and senior research fellow at Oxford University Bradford Saad published “On Birch’s ‘AI Consciousness: A Centrist Manifesto,’” an article that begins by agreeing with the seriousness of the two challenges presented by Birch but disagreeing with the notion that the manifesto addresses a “centrist” position.
From there, it generally agrees with Birch’s premises while acknowledging that there are many imperfect definitions in use when it comes to the general discourse surrounding AI consciousness.
Saad concludes that there’s more to the centrist position on AI consciousness than a binary division can encompass and advocates for broadening the scope of discussion. He writes that “getting reasonable debate into the information ecosystem makes it more likely that the spectrum for the politics of digital minds end up being defined relative to reasonable positions,” before adding that in lieu of such guided discourse, he expects “memetic evolution to summon a political spectrum structured largely by unreasonable views.”
Finally, over on the Less Wrong community forum, a writer named Oscar Gilg wrote a response to Birch’s manifesto from the perspective of an “illusionist,” or someone who believes our perceptions of consciousness are merely an illusion and that “phenomenal consciousness does not exist.”
Beyond semantics
Upon review, most of the responses to Birch’s manifesto appear to seek clarity and debate on the ideas espoused within. The collective reception, as seen on social media and in longform responses, could best be described as one that agrees with the basic ideas presented in the paper but struggles to understand why Birch sees them as urgent relative to other perceived threats presented by advanced AI.
Ultimately, as many of the responses pointed out, we already live in a world where millions of people are clearly confused about the capabilities of modern AI. It might be beneficial if the next steps for this vein of research involved defining the potential harm vectors associated with conscious AI and developing theoretical methods for counteracting them.
Ethical considerations:
- What are the harms of confusing simulated with genuine AI consciousness?
- Does a lack of consensus on consciousness prevent ethical AI development?
- Who should define the moral “harm vectors” for conscious AI?
Our newsletter bridges the gap between AI engineering, philosophy, and public debate. Sign up now.
A veteran AI journalist, Tristan was the creator and managing editor of The Next Web’s “Neural” imprint. He is one of the first and most prolific reporters to cover artificial intelligence as a full-time beat.