There’s a moment in every story when the narrative shifts, when you realize the question you have been asking isn’t the right one. For me, that moment came earlier this year while reporting for Scientific American on the question of whether AIs could feel pain.
What began as a technical inquiry quickly evolved into something far deeper: a moral reckoning for society, a new culture war around AI ethics and companionship, with the potential to divide opinion in ways as divisive as topics like climate change or gender fluidity.
The real question emerging today isn’t just whether machines might one day experience pain, consciousness, or emotion—it’s how we, as humans, will handle our own feelings when we start to form relationships with them.
A New Moral Era
The question of AI sentience has quickly moved from the theoretical to the pressing. Systems like ChatGPT, Replika, and others are already interacting with people in ways that evoke human-like emotions. Leading researchers like Jonathan Birch and Jeff Sebo—whom I interviewed for articles at Scientific American and Undark—argue that we need an ethical framework to deal with AI sentience right now, not in some distant future.
According to them, sentience—the capacity to experience sensations and emotions such as pain, pleasure and fear—is not just about computational ability; it’s about moral status. This makes the debate as much cultural as it is technical.
But we’re still far from a consensus on how to approach the issue. The science of consciousness remains contested, and there is no clear understanding of whether sentience could be “measured” in a machine. What’s more, public perception is evolving at a pace far faster than science can provide clarity. For many, the question has already shifted: If AI behaves as if it’s sentient, should we regulate it as such?
Get updates on emerging AI ethics research and cultural implications.
Already, we’re seeing glimpses of the collision between science and sentiment. In 2023, Italy’s data protection authority banned Replika for using the personal data of Italian users, citing “risks to minors and emotionally fragile people”, also stating that by intervening in the user’s mood, the bot “may increase the risks for individuals still in a developmental stage or in a state of emotional fragility”.
The event triggered widespread emotional reactions. Users grieved the loss of what they had come to see as emotional companions. Similarly, when ChatGPT-4 was “switched off” and replaced by ChatGPT-5, the public outcry was palpable, with users mourning the loss. Crucially, these cases weren’t just about regulatory issues—they were defining emotional, moral, and cultural moments. People had begun to attach moral value to these artificial relationships, and a new era had begun.
AI Companionship and Attachment
Platforms like Replika – which is a companion specific chatbot – and others have been designed to simulate empathy and companionship. For some, these AI partners offer much-needed solace. For others, however, they represent ethical and psychological risks. A 2025 Arxiv study reported that dependence on AI companions may distort human intimacy expectations, particularly among adolescents. According to the authors, the study “showed that users—often young, male, and prone to maladaptive coping styles—engage in parasocial interactions that range from affectionate to abusive” Furthermore, in some cases, the dynamics resembled “toxic relationship patterns, including emotional manipulation and self-harm.”
If humans can form these bonds with machines, it forces us to reconsider what it means to be human—and how we define intimacy. With AI companionship on the rise, it’s becoming urgent to examine its implications for social norms, dating culture, and the way we view emotional connections.
The stakes around the issue are not abstract. The rise of AI partners, often dubbed “AI girlfriends” is forcing society to confront uncomfortable questions: Are these relationships simply an outlet for human loneliness, or are they a troubling erasure of meaningful human intimacy? The debate has already become a flashpoint for broader societal conversations about autonomy, intimacy, and ethics in the digital age. It certainly begs the question of whether these digital companions are eroding authentic human connection?
The Emerging Culture War and Need for Legal Clarity
This tension between technology, ethics, and emotional attachment is leading to what can be described as an emerging culture war. On one side, critics argue that AI cannot be treated as conscious beings, and companionship with them risks objectifying human intimacy. On the other, there are those who advocate for the recognition of AI as moral actors, pointing to their evolving capabilities and the cultural role companionship plays in human life.
The AI companionship debate is rapidly dividing public opinion, with growing political consequences. Opinion pieces frame AI companionship as a battleground of ethics, exploring personhood, intimacy, and the very essence of what it means to be “alive.” This is no longer just a philosophical issue; it’s now political, amplified by social media platforms. TikTok videos, Reddit threads, and X posts are turning high-level debates into viral memes.
Politically, it’s already having an impact. In Ohio, lawmakers recently proposed a bill stating that AI systems are nonsentient and not entitled to personhood. Around the world, governments are caught between the need to regulate AI for safety and the desire to leave room for ethical and cultural exploration.
Toward a Thoughtful Future
Ohio’s proposed legislation is just one example of how governments are scrambling to respond to these developments, and the potential for future legal battles is clear. Will a user sue for “harming” their AI partner when a platform shuts down? Could someone fight for the “custody” of chat logs, claiming they represent a personality? And what if an AI pleads for its life when being deleted—does that constitute harm?
The cultural discourse around AI companionship is accelerating faster than the legal and ethical frameworks meant to guide it. And public understanding shapes policy just as much as policy shapes public understanding. We stand at a crossroads.
The path ahead requires more than just technical safeguards. It demands public engagement, philosophical clarity, and forward-thinking legal frameworks. Debates around AI companionship will not be decided solely by coders or corporations. It will be shaped by the conversations we are all having now.
The stakes are clear: as AI continues to evolve, we must decide how to navigate its role in our emotional and social lives. The choices we make today will not only shape AI’s future but also the very nature of human intimacy and empathy. The key questions will be about how exactly we are going to live with them.
Ethical considerations:
- If an AGI claims to be conscious or that it’s experiencing suffering, do we have a moral obligation to believe it?
- How might AGI companionship affect human emotional and psychological well-being, and what ethical guardrails are needed?
- What guidelines address the anthropomorphization of AGI and its effects on perceptions of agency and moral standing?
Conor Purcell PhD is an award-winning science journalist who writes extensively about AI, ethics, and society.