A team of researchers from Princeton recently published pre-print research advocating for a shift away from the “generalist LLM monolith” associated with developing AGI models. Their work advocates for a dynamic ecosystem where swarms of domain-specific superintelligences provide AGI-like capabilities at a fraction of the cost in energy and environmental impact.
The big contribution from their paper, An Alternative Trajectory for Generative AI, appears to be a sort of blueprint for removing our dependency on massive language models that are becoming increasingly power-hungry as demand shifts from training cycles to inference — generating answers to prompts.
The researchers argue that modern LLMs such as OpenAI’s ChatGPT and Anthropic’s Claude are environmentally unsound and unsustainable for reasons beyond just their energy impact:
“The prevailing trajectory, i.e, pursuit of artificial general intelligence through continued scaling of monolithic models is colliding with hard physical constraints: localized grid failures, prohibitive water consumption, and diminishing returns on data scaling. Whereas this trajectory yields models with impressive factual recall, it struggles in domains that require in-depth verifiable reasoning. A possible cause is a lack of sufficient abstractions in the underlying training data.”
Basically, the team claims that modern LLMs are struggling with coherence and failing to exhibit general intelligence because they’re not being given strong enough data.
We can think of a massive large language model as a big straw. When you prompt it with a query such as, “how close are scientists to developing a universal quantum computer?” it uses that big straw to conduct a process called machine learning inference to load pre-trained information in response to a query.
Unfortunately, there’s always a chance it’ll suck up information that’s factually incorrect or that the data will be insufficient to answer the query and the model might hallucinate something that’s not explicit to its training data during this inference process.
The Princeton team suggests that, rather than pursue the ever-scaling LLM paradigm to maximize generality, we should shift toward creating domain-specific superintelligences (DSS) built with symbolic abstraction and bespoke training for the purpose of “task-relevant sufficiency” and long-term sustainability.
What?
Let’s take a step back here and examine what that means. The scientists are challenging the narrative that “scale is all you need” to develop AGI and suggesting an alternative to LLMs.
The general reasoning behind the theory that “scale is all you need” is based on the hypothesis that an LLM of sufficient scope and scale will manifest human-like reasoning behaviors.
If GPT 5 has 2 trillion parameters, then GPT 6 should have 20 trillion or 200 trillion parameters. Under the auspices of this theory, this is how LLM developers demonstrate progress toward AGI.
Admittedly, it’s nowhere near that simple. There are hundreds of other factors that dictate how capable a given model will be. But, for the purposes of this discussion, this is the monolith that the Princeton team is referring to.
And, generally speaking, the energy and infrastructure costs rise in lockstep with the size of models — though there are mitigations.
What the Princeton team argues for instead is an entirely different approach to developing advanced AI. Instead of an LLM with a big straw trying to brute force its way into becoming an AGI model, they’re saying we should develop several “superintelligences” built on symbolic reasoning and bespoke data.
In their words:
“We argue that to achieve robust reasoning in open-world domains, it is advantageous to first construct explicit symbolic abstractions, such as knowledge graphs, ontologies, and formal logic. These abstractions can form the basis for generation of synthetic curricula, training on which can enable small language models to master complex domain-specific reasoning, without encountering the model collapse problem that is typical of current synthetic data methods that use an LLM to supplement training data for next-generation LLMs.”
The paper continues, “Rather than training a single generalist giant model, we envision ‘societies of DSS models’: dynamic ecosystems where orchestration agents route tasks to distinct DSS back-ends.”
Their vision is described as developing a family of specialist models representing measured expertise. The team says this paradigm is not only highly scalable, but it mirrors how human societies are organized.
“Individuals acquire expertise in narrow domains and cooperate to solve complex problems that require inter-domain expertise,” they write. “This idea has deep roots in cognitive science: Marvin Minsky’s Society of Mind framed intelligence as the coordinated effort of many smaller agents, and work on distributed cognition highlights how cognitive capacity emerges from interactions among people, artifacts, and institutions.”
Symbolic AI with a new wrapper?
The logical question here, for those of us who’ve been following the debates between Gary Marcus (who advocates for a symbolic AI approach to advanced AI) and many of the luminaries in the field of generative AI who believe that “scale is all you need,” is to ask how the Princeton team’s work differs from Marcus’.
The short answer is that they both agree that deep learning alone is insufficient for human-level reasoning. Where they differ is in the implementation. While Marcus suggests augmenting deep learning with symbolic AI to achieve a more robust AGI, the Princeton team proposes a decentralized approach that eschews the pursuit of a ‘generalist’ model altogether in favor of specialized expertise.
Ultimately, this work stands out as a third option. In my opinion, it distinguishes itself from Marcus’ hybrid approach and appears to embrace a development philosophy more rooted in sociology than other contemporary design paradigms.
It also has the potential to lend credence to the notion that a distributed intelligence, rather than a centralized (monolithic) artificial general intelligence, could be the most efficient path forward for superintelligence and equivalent advanced AI technologies.
While the research is promising, the team does flag three specific questions that remain open:
Q1. What minimal set of capabilities actually distinguishes general intelligence from a portfolio of narrow skills?
Q2. How should we trade off breadth and depth of an AI model?
Q3. How do we reliably separate genuine generalization from training-data leakage at scale?
At the end of the day, we’re still stuck holding a couple of bags no matter what paradigm we embrace. Whether we’re talking a monolithic AGI or a swarm of superintelligences, we still have to determine how, exactly, to ensure the machine is capable of moral agency and acting as an ethical operator.
And we still have to decide who gets to determine the machine’s morality and ethics.
Ethical concerns discussed in this article:
- Can AGI be designed to minimize environmental impact including carbon footprint, resource consumption, and pollution?
- If an AGI’s embodiment is distributed across networks, how can we attribute specific experiences and responsibility to distinct agents?
- How do we confirm semantic understanding in AGI, as opposed to mere data processing or mimicry of understanding?
Get sharper analysis on AGI ethics
Original essays, expert commentary, and curated analysis on the human stakes of AGI.
A veteran AI journalist, Tristan was the creator and managing editor of The Next Web’s “Neural” imprint. He is one of the first and most prolific reporters to cover artificial intelligence as a full-time beat.