AGI Ethics News asked experts across a variety of fields to pen essays discussing how modern AI tools and the potential emergence of AGI might affect their professions.
We learned that, by and large, we all have similar concerns. We’re worried that AGI will be smart like us, but that it won’t understand what makes us human and why those qualities are important.
Our hopes are also similar. We all want AGI to be a collaborative partner that can lift up humanity. Whether we’re dreaming of a future where machines help protect what he hold most dear, one where they help us reach our creative potential, or both, it’s clear that we all want to make sure AGI is on team human.
In this article:
- Welcoming AGI as a Creative Peer in Media Storytelling by Toni Williams
- Unless AGI possesses emotional intelligence, it will remain a tool by Wendy Ward
- Transparency is non-negotiable by Charlotte Kent
- Will AGI cover your six? by Frank DiGiovanni
Toni Williams is the visionary founder and executive producer behind Brooklyn Savvy, a pioneering media brand that amplifies diverse voices and sparks courageous conversations on the most pressing social issues of our time. Now airing its 21st season on NYC Life (Channel 25), Brooklyn Savvy is also distributed nationally on PBS.
In addition, Toni created and hosts Art Movez, a radio show and podcast that explores art at the intersection of social justice, technology, and innovation. Art Movez airs Sunday nights at 8:00 pm and Tuesdays at 12:30 pm on WNYE 91.5 FM.
I represent the world of independent community media production, where storytelling is more than content creation — it is a tool for social justice, healing, recognition, and community power. For close to two decades, I have created content for two platforms, Brooklyn Savvy and Art Movez, where I lift voices that are too often sidelined in mainstream media narratives.
The idea of welcoming a robotic AGI into this sacred creative circle requires me to pause and ask: can this machine truly honor the humanity — the embedded spirituality — of the work?
The critical questions for me start with transparency, values, and intent. Whose values shaped this AGI, and what biases are embedded in its algorithms? Can it distinguish between amplification and appropriation — between honoring a community’s story and exploiting it? Media narratives shape public opinion; my stories carry weight.
Will this collaboration generate thoughts that uplift or harm? I would need assurances that the AGI is committed to benevolence, that it would not generate work that retraumatizes marginalized communities or reinforces harmful stereotypes that would undermine the core of my mission.
Integrating AGI into my creative practice could accelerate research, widen access to archives, or perhaps reveal connections across disciplines, but would it add the level of nuance that is needed in masterful storytelling?
However, the AGI can be an incredible lure when one is constrained by financial resources that make it difficult to attract top talent. But not without some trepidation, as authenticity is a central ethos of the work I do. Authenticity would need to be balanced with the efficiency the AGI provides.
The heart and soul of my work lies in building trust, sitting with lived experience, and listening deeply — qualities no machine can fully replicate. The ideal relationship would be one where the AGI supports but does not replace the human heartbeat at the center of storytelling.
From the “AGI Trust Table,” the three most essential traits for me are transparency (I must understand its creative logic), benevolence (a clear alignment with community well-being), and validity (assurance that facts, histories, and cultural references are accurate and responsibly sourced and handled with care).
Co-ownership should not be a question, as an AGI does not have legal standing — or does it? If an AGI helps craft an episode, is it a co-creator — or a tool, like a camera or editing software?
For now, I lean toward a tool, a machine that captures my thoughts and can access the thoughts of many others, which makes it a formidable partner. Authorship is not just about generating words or images but about holding accountability for impact. Accountability cannot be outsourced to a machine.
At this moment, I hold both caution and curiosity about the emergence of AGI. Caution, because storytelling is too precious to risk dehumanization. Curiosity, because if developed with transparency, benevolence, and validity, AGI could widen the aperture of imagination. But the heart of the work must remain human.
Wendy graduated from Denison University with degrees in Psychology and Spanish. She was a successful sales executive before founding futuresTHRIVE, which is modernizing mental health screening and driving attention to youth health concerns. She is the winner of the 2019 HAYVN Hatch award and the 2025 Realist Lab Spark Impact award.
As the founder of a modernized mental health screening company, one of our jobs is to analyze the benefits of various technologies in monitoring, tracking, and treating kids with mental health issues – similar to hearing and vision screening.
We are infinitely curious about how we can leverage technology and not lose the unique benefit and nurturing effect of human contact.
Childhood development is a unique time of growth and change. In 18 years, the rapid transformation is complex both physically and mentally. Factors of family history, environment, family stressors, lifestyle, social/cultural expectations, and the unending combinations of human experiences that make each human unique, increase the complexity of development during these formative years.
With that in mind, there are substantial bodies of research that show how mental illness presents. If one has concerns, these key questions need to be addressed:
Intensity: How intense are the behaviors, thoughts, or emotions?
Frequency: How often does a child feel or behave this way?
Duration: How long do these individual episodes or periods last?
Functionality: Above all else, how well is a child functioning in life? Is your child impaired in any way at home, at school, or with friends?
Imagine we utilize AGI to support a therapist in determining treatment for a teen found to be struggling. In its current form, significant deficits present.
First, by nature AI is logic-based. Human development is not logical. AGI would need to recognize and navigate the complexities that underlie a child’s development.
Second, AGI lacks emotion, an understanding of emotion, and has already been shown to carry cultural biases. AGI would need to be able to truly recognize and interpret human emotion. For example, is a person crying because something is so funny, or so sad? AGI would need to factor in culture, values, beliefs, all the contextual elements that contribute to forming a young mind.
Third, AGI lacks perspective. For example, is mom telling me to journal my feelings or is my writing professor telling me that? Without perspective, any information can be misinterpreted. Finally, AGI would need to properly interpret nonverbal communications, which is additionally impacted by culture.
Essentially, for AGI to act as support to a therapist, it would need uniquely human qualities. Anything short of these capabilities, AGI would likely produce treatment plans and support that falls short or even fails. Evidence of this over the past five years is the growing number of kids (and adults) using AI chat bots for mental health support and taking their own lives. Additionally, we can look at companies like Mindstrong, that promised to use AI to develop cell phone usage biomarkers to detect suicidality, but couldn’t prove the technology worked and ultimately went out of business.
There is no doubt AI and AGI can crunch more data faster than humans, which is helpful. However, unless AGI specifically can be built with emotional intelligence and nurturing, it can only serve as a support tool alongside a clinical team, which could offer value if used accordingly.
Charlotte Kent, PhD, is an arts writer based in New York City and Associate Professor of Visual Culture at Montclair State University. She writes a monthly column on Art & Technology for The Brooklyn Rail, where she is also an Editor-at-Large, and is the recipient of grants from the NEH and Google’s Artist + Machine Intelligence program.
The term “artificial intelligence” was coined at Dartmouth, but researchers at Rand and Carnegie Mellon University preferred “complex information processing.” How much clearer that term might have been. Perhaps AI’s deceptive register stems from a misnomer on so many levels, besides a general forgetfulness that this thing “AI” was conjured from a particular culture, country, and context.
Artists that interest me have been questioning this compulsion to build “agents” and what values we ascribe to them.
I used to consider adopting the phrase “augmented intelligence,” but that no longer satisfies. This whole moment seems like a fleeting opportunity to consider what we value in human capabilities, which are probably not defined by nor limited to reason, logic, intelligence — after all, other times promoted other values.
The discourse around replacement outcasts some work, traits, and relations, as if these did not have aspects that contribute to the social fabric in ways being ignored. The task at hand is the configuration of new relational — not just rational — covenants.
AI channels imagination toward a future, distracts from the past, and enables a specific set of drives around the present. Apocalyptic fantasies of Artificial General Intelligence (AGI) — a term that forecasts an advanced machine with the same learning abilities as a normative human and therefore challenges human superiority — project a problem into the future while ignoring the supremacist systems rampant in our present.
Since large “reasoning” models (what I call LM sets) are known to reproduce and exacerbate inherent societal biases, scholars warn against creating fully autonomous agents for very good reasons. Whether advances will ever satisfy the fantasies surrounding AGI seems unlikely, as the goal always shimmers beyond whatever we accomplish.
To consider accepting them, I would need to know precisely what value systems and implicit assumptions proliferate around each AGI — and that is not possible by the very design of machine learning models. But if pressed, my foremost need for an AGI collaborator would be transparency regarding its “set”: the underlying data sets, the code, the algorithms, and the culture, country, and context from which the technology has been conjured.
Transparency is non-negotiable; without understanding the background process, one cannot “face the AI.” Confusion regarding what one is observing — the complex system of the LM set—leads to unease, often mistaken as outrage against a liar.
We must distinguish the AGI’s complex “distinct agentive forms” to move forward. I think the comparison to humans ignores the likely necessity of more complex negotiations around what we think agency is, including the likelihood that it isn’t singular.
At this moment in the development of AGI, I feel a necessary combination of anxiety (fear and repulsion flow easily) and mobilization. The humanities are notoriously ever in crisis and so they offer a model for enduring and adapting through doubt, debate, and defending value.
I expect in my lifetime to see things get worse in human relations to each other, the planet, as well as whatever newfangled computational creations we devise.
Reflecting on these technological possibilities, though uncomfortable, keeps open the opportunity to configure new paradigms, which we self-evidently need. I maintain that effort even though I see these technological developments building off old models of thought, usurping the opportunity for real change.
Dr. Frank “D9” DiGiovanni is Chief Growth Officer at Aries Security and CEO of D9Disruptioneering, a consulting firm focused on AI meta-design, cybersociology, and organizational transformation. He brings forty-five years of experience across the military, national security, and private sectors.
In his national security work, he researched the soft-skill attributes — what sociologists call habitus — that distinguish elite cybersecurity operators from their peers, and developed a novel framework to identify these attributes in new recruits and deliberately cultivate them throughout their careers. He holds an Ed.D. (double distinction) from the University of Pennsylvania.
Estimates for when AI will match or exceed human performance across most cognitive tasks — the benchmark for Artificial General Intelligence (AGI) — range from around 2030 to several decades out. Given the current rate of progress, the 2030s seem more plausible than the distant future.
If AGI is that near, three questions must be resolved before admitting it as a collaborator in the national security sector:
Safety, security, and governance. What safeguards bound AGI behavior, and what exactly occurs — technically and organizationally — if it is compromised? Engineering must meet risk-based standards with explicit operational constraints, interlocks, audit logs, incident response, and red-team testing.
Use of two complementary risk constructs is essential: fail-safe (classical engineering that defaults systems to a safe state for foreseen failure modes) and safe-to-fail (from Dave Snowden’s Cynefin framework, which employs small, bounded experiments to probe the situational environment and observe its response). Positive signals are amplified; negative signals are dampened or terminated.
In complex contexts where outcomes cannot be predicted, these probe–sense–respond cycles enable the human–AGI team to chart a viable path forward — even when no precedent exists. The objective is rapid learning with tightly contained failure.
Anchored to the ethos of “to boldly go where no one has gone before,” the AGI governance construct must deliberately stimulate collaborative, novel, and innovative thinking as it will operate within an inherently adversarial national-security context. Predictability, rigid reliance on standard operating procedures, and unexamined adherence to the status quo create exploitable patterns.
The AGI’s governance should be bounded by two opposed but parallel guardrails: on one side, strict adherence to structured decision processes (such as the U.S. Army Military Decision-Making Process) to ensure rigor, traceability, and risk control; and on the other, institutionalized adaptability that normalizes hypothesis-driven challenges to assumptions — always questioning the norm. The norm is not presumed invalid, but its continued use should be data-conditioned and evidence-tested.
Accordingly, the AGI should conduct explicit metacognitive self-checks and employ Socratic questioning with its human partner to surface alternatives, validate premises, and recalibrate when indicators show a superior course of action.
Reliability and validity. How accurate, valid, and consistent is the AGI under real operating conditions? Hallucinations, incorrect outputs, opaque reasoning, and inconsistent behavior can lead to mission failure and loss of life. It must be rock-solid in this regard — verified against task-relevant benchmarks, stress-tested on edge cases, monitored for position drift, and sufficiently transparent to trace decisions. Transparency may remain a demanding goal but is possibly feasible within the next generation of AI technology.
Human–AGI co-evolution. Is there a deliberate mechanism to ensure the human and the AGI co-evolve — cognitively and affectively? The relationship must be symbiotic, preserving and strengthening human agency. For any topic or decision that requires Keith Stanovich’s reflective or algorithmic mind, the AGI and the human must engage the higher thinking processes of both brain and system, invoking metacognitive thinking to stay adaptive and valid while Socratically stimulating its human partner.
Users are increasingly deferring their thinking to AI. As Stanovich argues, humans are cognitive misers; even when capable of analytic reasoning, we often fail to engage it. AGI can amplify that “lazy mind” unless we design for reflective override, calibrated trust, and continual human skill growth. Humans and AGI should be interdependent, each pushing the other to improve.
In national security, we will admit AGI only when risk is engineered for both the known and the unknown, novel and innovative solutions are not dampened or discredited, operators can trust the AGI has their six, and it demonstrably sharpens — rather than dulls — human judgment. Otherwise, it stays outside the wire.
In AGI Opinion we ask experienced leaders. managers, and scholars to weigh in on issues that are pressing or revealing for AGI design and development and aligned with our Focus.