For decades, the search for artificial intelligence has centered on scale. The prevailing wisdom is that bigger models, more data, and ever-increasing compute would provide the foundations for machines that can do everything. But an alternative approach is emerging from research in multi-agent systems.
The goal is to move beyond a single monolithic model and instead build networks of smaller, specialized agents. By working together, they could add efficiency and control to the path toward AGI.
Small language models (SLMs) offer a potential set of building blocks for this transition. Individually, these lightweight systems are limited. But as a group, they can spread cognitive tasks across multiple components, rather than burying them inside one opaque, enormous model.
This would mark a shift from recent landmarks on the route to AGI. DeepMind’s Gato, for instance, demonstrated that a single model could be trained on a wide range of tasks, from playing video games to controlling robotic arms. The system was a landmark generalist agent, but rarely matched specialized models in their specific domains.
The multi-agent SLM approach starts from a different premise. Instead of concentrating capabilities in a single model, it distributes them across connected systems that can work together. Together, they begin to resemble something close to a collaborative society.
From one model to many
This structure offers promising efficiency improvements, as smaller models require less compute and energy, and may offer a more sustainable way to scale AI. It could also make behavior easier to monitor and adjust, since individual parts can be tested and controlled more directly.
Dr Stefano Albrecht, an associate professor at NTU Singapore who researches multi-agent systems, argues that this approach has advantages over a single, all-powerful model.
“A large, intractable problem may become tractable if we can divide it into smaller sub-tasks that can be efficiently solved by specialized AI agents,” he tells AGI Ethics News via email.
“For example, training and deploying a set of specialized small language models (each with millions of parameters) for certain applications may be more efficient than using large language models that have many billions of parameters and are expensive to deploy.”
“The faster inference times of smaller models (e.g. milliseconds rather than seconds) may also be critical for applications that require real-time responses.”
But breaking intelligence into smaller parts also introduces the challenge of coordination. Multi-agent systems must decide how to divide tasks, share information, and align their actions in real time. Albrecht points to two separate solutions: multi-agent reinforcement learning and hierarchical structures.
“Multi-agent reinforcement learning (MARL) enables agents to learn and co-adapt in order to produce highly effective coordination strategies,” he says. “Other approaches involve using hierarchies in multi-agent teams, so that some agents assume leadership and management responsibilities to improve teamwork coordination.”
Early versions of this are already appearing. Some systems use “manager agents” to orchestrate others, assigning tasks, monitoring outputs, and adjusting strategies in real time. The architecture begins to resemble an organization rather than a single tool.
In systems built from SLMs, this kind of coordination could combine their capabilities into more general intelligence.
Distributing intelligence
At first glance, this architecture may resemble distributed machine learning, where large models are split across multiple machines to improve performance. But distributed systems still operate as a single intelligence. They optimize only one objective, even if computation is spread across hardware. Their components do not act independently.
Systems built from SLMs work differently. Each element has a defined role and a degree of autonomy. Instead of one objective, there are many individual goals, aligned but not identical. That creates a different approach to a concept at the center of the AGI debate: agency.
To perform any cognitive task, AGI must have the capacity to act across domains, over time, and in response to changing conditions. Systems built from SLMs aim to provide this by having each model operate within a narrow scope. They then combine their specialized capabilities across domains, coordinate actions over time, and adapt by adjusting individual components rather than the whole. General intelligence emerges from the interactions between them.
This approach mirrors how complex systems operate. In organizations, markets, and teams, intelligence is distributed. No single part understands the whole, yet coherent behavior still emerges from interaction.
Editor’s note: Interestingly, this interaction-based emergence paradigm is also how computations are derived in photonic quantum computers.
Whether artificial systems can achieve this reliably remains uncertain. But these models of distributed intelligence suggest that AGI may be less like a single mind and more like a coordinated system.
The trade-offs
Pursuing AGI with SLMs presents both opportunities and challenges. On one hand, smaller models may be easier to understand and control. Individual components can be tested, audited, and aligned with specific goals. Failures may also be more contained than in a single system.
On the other hand, distributing intelligence introduces new uncertainties. Emergent behavior becomes harder to predict. Coordination failures could lead to system-wide issues that are difficult to trace back to any single component.
There is also the question of control. Systems that rely on coordination layers or “manager agents” may concentrate influence in specific parts of the architecture. Ensuring those components remain aligned could prove as difficult as aligning a single large model.
There is still little direct evidence that networks of SLMs can achieve general intelligence. Most leading AI systems rely on large, centralized models, and the challenges of coordination remain unresolved. But smaller models are becoming more capable, and research in multi-agent systems is advancing quickly. Together, these trends are making the idea of AGI as a society of collaborating agents harder to dismiss.
Ethical concerns discussed in this article:
- What criteria ensure that the AGI’s recurrent processing does not merely create fragmented information, but coherent subjective experience?
- If an AGI’s embodiment is distributed across networks, how can we attribute specific experiences and responsibility to distinct agents?
- Can AGI be transparently audited and explained so its decision-making processes are comprehensible and accountable to humans?
Thomas is a freelance tech reporter who has covered AI for nearly a decade. He was previously managing editor of The Next Web, a Financial Times company. In his free time, he likes playing chess (badly) and basketball (even worse). You can reach him at thomas.s.r.macaulay@gmail.com