What if an AGI worked as well as an automated call answering system? Disaster. Even though they are smarter than yesterday, our everyday communications technologies—dropped texts, garbled calls, frozen meeting screens, and notoriously difficult to navigate user interfaces and call answering systems—are painful and indifferent. Fixes are greedy with our time.
As software developers, for a community of 10,000 users, my brother and I invested in 200 beta testers. With some 168 products, we had one technical release (fix) in 15 years. This is not a matter of pride; it’s an acknowledgement that technology is a design challenge that can only be met with humility. OpenAI released a free version of ChatGPT and, without saying so, had more than 100 million beta testers and still has not solved problems of anxiety, burnout, and sleep disturbance, sycophancy, factual accuracy, logical consistency, and more.
Software developers are a cult, not in the sense of a conspiracy, but in the sense of following certain tools, certain design practices, certain processes of convenience and thrift, and certain numerical assumptions about complex behaviors, instead of being resolutely user-centric. Even with the best of intentions, this is a recipe for pathology.
The cult often makes assumptions about the layers of understanding required for ease of use, or harmony, with our technology. Take the simplest example, getting from A to B in your apps: If you use 10 apps in a day, each with 5 screens, each screen with 10 options, each option with 3 layers or decision points, then you’re expected to successfully navigate 1500 times a day. The cost of poor user interface—of poor design—is sucking up user bandwidth, or worse, more direct harm.
Successful software design is domain specific: you must speak the language and know the tools, practices, and habits of your user—altogether what Heidegger called “equipment.” Equipment are [sic] bounded horizons of expertise that are at their best when one’s mastery requires little or no overt thinking: for instance, when you’re opening a door or brushing your teeth, playing in an orchestra or twisting on a balance beam.
AGIs will never be just a tool; they will be, at the very least, integrated equipment, in that broad sense—part of a bounded domain of expertise that requires more than use, one that requires cultural sensitivity and mastery. Is this a surprise? For so do human relations—they require years of time, attention, skill, dogged perseverance, and forgiveness.
Working with domains, the sensitive designer must respect cultural differences, if only to discover workable universals. Universal design is an aspiration for the diligent. But kids are a culture. Left-handed is a culture. Blind is a culture. Financially literate is a culture. So for most builders seeking to design for millions, there will always remain a necessity to design adaptable, bespoke items and experiences for different communities—to design what is intuitive, relevant, and appealing for different abilities and psyches. Especially if the aspiration is trust.
Want to explore how ethics and culture shape AI design? Subscribe to AGI Ethics News for exclusive insights and updates.
Trust is a kind of collaboration, and at AGI Ethics News, we take it as a design principle that AGI, if it is to arrive harmoniously in our world, must be trustworthy. We must collaborate with these new AGI beings, whether their body is a box, an accessory, an implant, a distributed matrix, or a humanoid. Collaboration has its own language and is subject to cultural differences.
Why is this important? Because an AGI must fit into our lives, not incidentally, without disrupting human interaction.
As Bruno Latour writes in Pandora’s Hope, commented on here by Brian Christian in The Alignment Problem, “We have taken science for realist painting, imagining that it made an exact copy of the world. The sciences do something else entirely—paintings too, for that matter. Through successive stages they link us to an aligned, transformed, constructed world.’ Aligned—if we are fortunate, and very careful, and very wise.… We are in danger of losing control of the world not to AI or to machines as such but to models. To formal, often numerical specifications for what exists and for what we want… The best model of the world stands in for the world itself, and threatens to kill off the real thing.”
This is a dark outcome. It will take deliberation and creativity to avoid.
At his talk before The Royal Society this month, Gary Marcus posited “The very idea of artificial general intelligence—AGI—might not be the right goal, at least not right now.” This is not the first call for deliberation—just last month Luciano Floridi powerfully reinforced the call for ethical design saying “Ethical governance is not a brake but a steering wheel…” Good design is ethical. Good ethics requires human imagination and integration, and that takes time.
LLMs—and cell phones and even answering systems—are still working out interface and control design. Are we ready for systems as complex and nuanced as artificial general intelligence? Our newsletter focus this month was originally “Safety, Control, and Alignment,” but isn’t it more helpful to elevate the design aspiration of “control” to collaboration and trust?
Join a community dedicated to responsible AGI integration. Sign up for our newsletter and stay informed on AI design and ethics.
K.B Miller is a lifelong publisher and writer with a degree in Logic and Philosophy of Science. He served on the Smithsonian National Board, where he co-founded the Education Committee; is Chair Emeritus, Cooper-Hewitt, Smithsonian Design Museum; and was principal investigator on a two-year AI research project for the U.S. Department of Defense.