America’s largest tech companies have spent $155bn on AI-related capital expenditures in 2025. It’s arguable that the biggest factor driving this investment isn’t what AI can do now, but what it might do in the future.
Depending on who you believe, we’re either on the precipice of creating artificial general intelligence (AGI), or it remains a distant, theoretical dream — one that lacks any concrete, agreed-upon definition.
In January of this year, OpenAI CEO Sam Altman said his firm is “now confident it knows how to build AGI as we have traditionally understood it.” Both Altman and Dario Amodei, the CEO of Anthropic, have posited 2027 as a likely arrival date. Conversely, Yann LeCun — Meta’s Chief AI scientist — believes that the development of AGI will be, itself, gradual, arriving not with a bang but a crescendo. He has also stated that large language models (LLMs, the basis of tools like ChatGPT and Claude) will not scale to AGI.
Put simply, the emergence of AGI is, itself, still unknown. And it’s noteworthy that those who are most convinced of its imminence are also those running large AI companies best positioned to benefit from a sustained AI hype cycle, as they require constant inflows of investment capital to remain afloat.
If we suppose that Altman and Amodei are correct, we’re left with a troubling moral dilemma. Although AGI could be yet another business tool — albeit one that displaces human workers and drives down wages and working conditions — many respected AI researchers have argued that the technology could present an existential threat to humanity.
British-Canadian researcher Geoffrey Hinton — a man described as the Godfather of AI, due to his pioneering work on deep learning — has warned about the potential danger of militarized superintelligences that can “create sub goals” that are not aligned with the user’s intentions. In an interview with CBS News in 2023, he said that it’s “not inconceivable” that AI could “wipe out humanity.”
These fears have previously been echoed by Altman, who described the potential worst-case scenario for AGI as “lights out for all of us,” although he has since tempered his predictions, claiming that AGI will not be as impactful as once thought.
Australian moral philosopher Peter Singer has, throughout his career, explored one’s moral obligation to act in response to suffering and harm. One of his thought experiments describes a drowning child in a pond, where an onlooker has to decide whether to save the child at the cost of being late to an important meeting, or perhaps muddying their most expensive clothes.
Any rational person would, in that situation, intervene with the innate knowledge that the worth of that child is greater than any outfit. From there, Singer extrapolates to the broader issue of global poverty. Instead of sacrificing a single outfit, would you live a simpler life if it meant you could donate more? Singer would argue that the ethical position is to make those sacrifices.
I mention Singer for another reason — in part due to his influence on the direct action wing of the animal welfare movement, which peaked in the early 2000s, and was defined by headline-grabbing organizations such as the Animal Liberation Front (ALF), Stop Huntingdon Animal Cruelty (SHAC), and the Animal Liberation Brigade.
In 1975, Singer published Animal Liberation, which described the conditions in research laboratories and farms in vivid, emotive terms, which he described as “the tyranny of human over non-human animals.” As with his other writing, Singer encouraged the reader to take concrete actions that would minimize that suffering — such as foregoing meat and other animal products, or refusing to wear leather.
In his later work, he addressed the morality of actions undertaken by groups like the ALF, writing in Practical Ethics that “damage to property is not as serious a matter as injuring or killing; hence it may be justified on grounds that would not justify anything that caused harm to sentient beings,” especially if the cause — alleviating the suffering of non-human animals — is just.
Singer’s work is pertinent to the discussion of artificial general intelligence and its risks because it addresses both the moral obligation of an individual to act in response to the suffering of others, as well as the form in which that action may take (whether that be personal lifestyle choices, or the decision to take more radical action that breaks the law).
Prima facie, there is an argument to be made that if artificial general intelligence represents a potential existential threat to humanity — as stated by Hinton, Altman, and others — then taking action to curtail or stop the development of AGI is not merely justified, but also a moral imperative.
A recent article by Forbes describes several students at MIT and Stanford who have done just that, leaving their studies due to fears, as one person interviewed stated, because they were “concerned [they] might not be alive to graduate because of AGI.”
These former students dropped out of their courses to alleviate those perceived risks, with one respondent working for a nonprofit organization “examining deceptive AI systems that could act against human interests.” Others joined the AI industry, perhaps in an attempt to steer companies like OpenAI away from danger from the inside.
None have taken the kinds of direct action demonstrated by groups like SHAC or the Animal Liberation Front, although it’s worth noting that there is historical precedent. In the early 1980s, the French anarchist group CLODO (an acronym that, translated into English, means “Committee for Liquidation or Subversion of Computers”) launched arson attacks against technology companies over the belief that computers helped facilitate the exploitation of workers, and were a tool for government surveillance.
The students interviewed by Forbes are operating within the moral framework established by Singer, which stressed the importance of one’s individual responsibility to act, even if those actions are, themselves, small.
The separation comes when you consider the still-theoretical nature of AGI. We do not know whether AGI is, in fact, imminent, or whether the proclamations of its arrival are designed to benefit capital-hungry tech companies. We don’t have a concrete, established definition for AGI. Nor do we know what form AGI will eventually take — and whether it will be able to deliver on the worst fears of Hinton and Altman.
In the light of these variables, the question of one’s moral responsibility to take action becomes a lot more complicated.
If there’s a 0.1% chance that, by 2027, Silicon Valley will create a superintelligence that extinguishes humanity, do you have a responsibility to act? If the chance becomes 0.01%, or 0.001%, do you still have that obligation?
And after how many zeroes does that obligation disappear?
Ethical considerations:
- Do individuals have a moral impetus to stop harmful AGI?
- If so, at what scale does existential threat merit such action?
- What actions are justified when facing existential threats?
Matthew Hughes is a UK-based writer and journalist from the United Kingdom. His work has appeared in publications such as The Next Web, The Register, and Wired. He currently runs What We Lost, a newsletter about the excesses of the tech industry — and their human cost.