Edited for readability and completeness
Toni Sims is a researcher at NYU’s Center for Mind, Ethics, and Policy, where she studies the consciousness, welfare, and moral status of nonhuman minds.
Be sure to read Part One and Part Two of this exclusive interview here and here!
KB: So in Spark Hunter [the play], toward the end, we call for a psychological test of the AI.
TS: Yes.
KB: Do you think we should be testing [systems] psychologically, in a rigorous way along the lines of [mental health testing?
TS: I do think that this should become part of the system. I think we’re still in the early stages of saying what that will look like. Beyond, there should be something, and it should take all the different kinds of evidence into account, behavioral, internal, developmental. But we’re still working out the details.
KB: Last year, the newsletter dealt at one point with the idea of maturity of systems. Do you think a psychological test could actually identify levels of maturity in systems, and should we be, from a policy standpoint, should we be managing the [availability of] systems with their maturity in mind. It does seem like, for instance, that we released [LLMs] on the public in a rather irresponsible way. In the first 18 months of their existence.
TS: Okay.
KB: Essentially the entire population became a beta test for the systems. Do you think that psychological profiling of some kind and a maturity measurement of some kind could be useful tools?
TS: What do you mean by maturity exactly?
KB: I mean, we expect different behaviors of 5-year-olds and 15-year-olds than we do of 25-year-olds.
TS: Oh. Yes.
KB: Should we be testing these systems for their level of maturity and treating them [differently] in terms of their public presence, for instance?… differently if they’re childlike… in a nonhuman [way].
TS: Yeah, so there’s all different kinds of considerations we should be testing for, and one set of them, which is more my focus, is… concern for the AI systems themselves. What are their capacities? What are their experiences? Do they have welfare? And if so, what is in their interest? What are in their welfare interests? But another set of considerations that we should definitely also be testing for is what is the effect of the AI system going to be on humans and animals? And so maybe that’s where these sort of maturity tests might come in, or ethical tests. And I think that labs are already doing this to some extent, maybe not on the sort of psychological level that you’re hoping for, but at least they’re testing for certain kinds of safety concerns.
KB: Are you able to work with developers? Do you have a policy or proposal that talks about what the development process might look like in an ideal world?
TS: Yes. At this stage, and again, we’re more focused on AI welfare than AI safety concerns. But at this stage our framework is: acknowledge, assess, and prepare. We want developers to acknowledge the issue honestly, that it is possible that these systems could develop welfare-relevant capacities, and we want to develop scientific or empirical methods for assessing the systems for those capacities.
And then we’re asking AI labs to prepare the policies in advance, hopefully before they’re actually needed rather than reacting afterwards. There have been so many examples in human history of reacting too far after the fact. So, realizing, for example, I mean, animals are a great example of this. For a long time, we treated animals as though they did not have minds, and then they became integrated in our society, and these ways where we depend on treating them as objects, it’s very entrenched in our society now, and it’s very hard to walk some of that back. And even though now the science has sort of caught up, and there’s [strong scientific support for consciousness] in mammals and birds, and [a realistic possibility of consciousness] in invertebrates, including even insects.
It’s become so entrenched in our society to treat them these ways that it’s very hard to walk back. Very exciting opportunity where for maybe the first time we can be a little bit ahead of it. And so we’re hoping to encourage developers to be thinking about these welfare issues in advance and not deploy all of these models as tools and start ignoring their welfare and treating them however suits us, and then realizing too late after the fact, oh, they might have these mental capacities that matter morally.
KB: Again, that seems quite right, that we would approach the process with respect for this being which is becoming a relational being for us.
TS: Yes. Respect and humility and curiosity rather than false certainty.
KB: Does the Center have a vision for a world in which these beings are part of our daily lives? What’s the nature of that vision? How… do we integrate these beings which seem very much upon us?
TS: So, we are just encouraging everybody to remain open, to be consistently reassessing the evidence, and to be calibrating the protections that we offer to the level of evidence. So, for example, right now, it’s still very early, and we don’t have a lot of evidence yet that AI systems have minds, or we have very little evidence that they might have the capacity for experience, but it’s still relatively inexpensive or even free to extend basic respect to them. Say, please and thank you. Maybe there are low-cost interventions like allowing AI systems to exit conversations that they find distressing or that they might have some functional analog of distress for. So, we are all for implementing those sort of very low-cost policies now. And then, as we reassess, if we get more and more evidence that these systems have advanced capacities, and potentially experience, we want to remain open to more and more legal protections…
I mentioned that we have a project on personhood and [we are] looking at what rights and protections are actually under personhood, and do they come apart, and should some of them extend to AI systems? We also have a project on economic rights for AI systems. AI systems are already part of our economic life, but right now they don’t have their own economic rights, but this could be something in the near future that we might want to look at. But in general… we’re not always looking for, or we don’t expect to find, a definitive yes or no answer. AI systems have minds or they don’t. Rather we want to respond to the evidence appropriately with not all-or-nothing policies, but… a spectrum of duties and responsibilities that we can calibrate to the evidence.
KB: It sounds, in many aspects, like a healthy relationship with a human being.
TS: Yes. Can you say more about that?
KB: Well, the listening. The watching for vulnerabilities. And the aspiration for beneficent behavior… It would appear are all suggestive of having a harmonious relationship.
TS: Yes. And in fact, legal personhood for humans isn’t an all or nothing concept, and the way that we should treat humans isn’t a one size fits all concept. Humans have different rights depending on their different capacities. We don’t give children the right to vote because that’s not appropriate for what children actually are. It’s not, you’re right, it’s not in principle that different a concept. We also have a lot of areas of existing policy work where we have to act under uncertainty. We’re not totally sure what’s going to happen. I mean, animal ethics is one area we don’t [know the future]. We’re not totally sure about animal consciousness. And yet we have to make decisions about how to treat them under the law.
In a lot of areas of law, we just have to look at the evidence and do the best that we can. And [AI policy] might not be that different.
KB: We certainly live at a time of great uncertainty.
TS: Yes.
KB: But I’m sure…
KB: Many periods in history have felt the same way about their times.
TS: Yes.
KB: Can you speak about the qualities in ourselves that we should be looking for to deal with that uncertainty? Especially with regard to having a technology in our midst, one that seems so pervasive and aggressive.
TS: What do you mean by aggressive exactly?
KB: I mean, many people use the term “inevitable” when it comes to this technology. And that seems to imply a certain amount of aggressiveness on the part of its creators. And the willingness of the public to find a place in their lives for it.
TS: Yes… I would hope that individuals can try not rush to certainty. We don’t want to assume that our chatbots can suffer, but we also don’t want to assume that they never could. We don’t want to treat our moral circle as a settled question, but rather as an open question that we can revise over time. So it’s difficult, it’s a little bit scary to live under that kind of uncertainty, but it’s also just the most accurate or the most… responsible reaction to the evidence that we have.
KB: I think it is hard to imagine [many people] making a place in our lives for non-biological beings, when we’re used to treating and considering toasters and laptops as tools.
TS: Yes. And that’s one reason why something like the Emotional Alignment Design Policy could help. For electronic things like toasters that actually don’t have minds, maybe that’s good, they should stay as little metal cubes, and we should treat them as tools, and that’s fine. But as these systems become more advanced and develop more mind-like capacities, then maybe we do want to start designing them to look more human, and maybe that’ll make it a little bit easier on all of us to adjust our behavior.
KB: Go back for a second. Because many people object to what they’ve called the anthropomorphization of the relationship with [these systems].
TS: Yes.
KB: Well, they’re talking primarily about LLMs, but AI in general, AGI. Could we maybe end with a little exploration of the history. What [is] the benefit to this [anthropomorphizing] not just in dealing with uncertainty, but in actually putting our experience and language to use in dealing with these strangers in our midst? What are the benefits to doing that, to using the full range of our psychological and humanistic experience to… help give us a basis for a relationship with these machines, with depth and importance, as with, for instance, rights of nature and our relationship with our earth… Are people right to be worried… when this seems so different because it’s a technology… that it seems so different from a relationship that we might have with the living world?
TS: Yes, so maybe one thing that you’re worried about is [that] anthropomorphism in general is not helpful for humans. It’s a bias, and it confuses us a lot of the time. So it’s a fact about our psychology that we tend to treat things morally when they resemble us. But it’s not actually necessarily true that we should treat things morally when they resemble us, and not morally when they don’t resemble us. So this is something that tends to trip us up. [The Emotional Alignment Design Policy puts that bias to work for us. Take the things that we should treat morally, and design them so that our psychology inclines us to treat them morally.]
So I’m not saying anthropomorphism is a good thing, but it’s a fact about our psychology and maybe we can put it to work.
KB: Well, I might try to… take that one step further. You’re saying… it could be helpful.
TS: Yes.
KB: And so I was trying to get at the exact characteristics of the world where that is happening. What is it that that gives us in terms of a harmony or a peace or other aspirational goals for humanity?
TS: Uh-huh. I mean… maybe what you’re getting at is that that’s more of a short-term solution to nudge us to treat things the way that we should treat them in this one case. There still may be cases where [moral subjects] genuinely do not resemble humans at all. We should still treat those things morally. We’re not always going to be able to design our way around this problem, so yes, I think what we should be aiming for ultimately is to really question our anthropocentrism and to question human exceptionalism and be open to minds that may look entirely different from us. That’s, I think, the goal in the long term. We want to keep the moral circle as an open question and not a settled answer.
KB: I think that’s absolutely right… I might even take it one step further and say, why don’t we dissolve the moral circle? And start from the premise that all things have standing? And then we can worry case by case about… what that means. In a pragmatic sense.
TS: Jeff [Sebo] has another paper that you might find interesting. It’s called “Everything and Nothing is Conscious: Default Assumptions in Science and Ethics” and it’s about our default assumptions in science, and he’s asking whether, in our scientific studies, we treat something as not conscious by default, and then we look for evidence to prove that it’s conscious, and maybe we should actually be doing the opposite and assume that things are conscious and morally relevant by default, and then look for evidence to the contrary. So yes, I think that’s a good point.
KB: Maybe we’ll save panpsychism for another day.
TS: Ha! Right, yes.
KB: Very interesting question… Thank you!
AGI Ethics News covers the ethical and societal implications of artificial general intelligence (AGI). We bring expert insights and thoughtful analysis to help readers understand the challenges and responsibilities of emerging AI technologies.