The Second Explosion: Why Multi-Agent AI Defies Traditional Engineering

I have worked in software engineering for more than thirty years and witnessed several combinatorial explosions firsthand. The AI-driven explosion now under way is different, and that distinction should shape our response. In both software and agentic engineering—the extension of software engineering to ecosystems of software and autonomous agents—quality must remain the central concern. If code can now be produced at a staggering pace, what limits our ability to deliver high-quality products quickly? The bottleneck lies in the scaffolding surrounding construction. Writing code faster isn't the danger; the danger is what happens when that code starts talking to itself.

The desire to scale agentic engineering is understandable given how deeply software permeates society. At the time of ENIAC, few could have anticipated that software would enable fly-by-wire aircraft, self-driving cars, embedded medical devices, machines that surpass humans at chess and go, or unmanned missions such as the Mars rovers. The lesson is clear: modern society depends on software nearly as much as it depends on electricity. Because software had to run everywhere, we built tools to twist single codebases into infinite variations. And that is where the first explosion happened.

The first kind of explosion

In the C and C++ world, we had a beloved and terrible tool: the preprocessor. #ifdef, #ifndef, #define, #pragma. You could take a single source tree and compile it into wildly different programs depending on a pile of switches. Need a debug build? A build for a different platform? A build with a feature on for one customer and off for another? The preprocessor would oblige. It was expressive, it was convenient, and it was seductive.

It was also a trap. Every independent switch doubles the number of possible configurations. Ten switches are 1,024 builds. Twenty is a million. Thirty is a billion. Long before you reach the limit of what your build machines can churn through, you have reached the limit of what any human or any test suite can meaningfully cover. The code looks like one program. It is an exponential family of programs, most of which have never been compiled, let alone tested.

What made this insidious was that the explosion was invisible. You did not feel the combinatorial growth as you added a flag. You felt it six months later when a customer reported a bug in a configuration nobody had ever run.

Notice what kind of problem this is. The configuration space is enormous, but it is enumerable. You can, in principle, list every combination. It is a known unknowns problem: you know exactly what you are not testing. The industry's response was not to ban the preprocessor. We built build matrices. We built continuous integration across configurations. We built feature-flag discipline and, eventually, configuration-as-code with explicit per-configuration test coverage. The response was infrastructural, not legislative. That precedent is worth remembering.

A second instance of the same pattern

I was an early adopter of Scala, back when the language still felt like a miracle. You could craft domain-specific languages directly in Scala itself—parser combinators, implicit conversions, operator overloading, the lot. (An opinion I wrote about an early edition of Programming in Scala even made it into the book's website.) The expressive power was intoxicating.

Without conventions, it was also a maintenance nightmare. Every team, sometimes every developer, invented a micro-DSL. Each was locally elegant. Together they were a Tower of Babel: a codebase where >>, ::, ~> and | meant six different things in six different files, and where the meaning of a line depended on which implicit conversions happened to be in scope. The explosion here was not an exponential number of build configurations—it was a combinatorial explosion of vocabularies and their interactions.; every pair of micro-DSLs had to be understood together.

And the response was the same: not prohibition, but convention. Style guides. "Don't do this" lists from the community. Library design guidelines. Eventually tooling. We kept the expressive power and built the discipline around it. Control was restored—not by removing the mechanism, but by agreeing on how to use it. So, the pattern repeats: a genuinely powerful way to express intent; no conventions at first; a combinatorial explosion no one can hold in their head; then, after the pain, a body of practice that brings it back under control. The preprocessor taught us this. Scala re-taught it.

But notice what both cases have in common. The explosion, however large, was enumerable in principle. You could list the build configurations. You could, with enough patience, catalogue the micro-DSLs and their interactions. The problem was scale and discipline, not knowability. That is the first kind of explosion, and it is the kind that practice can fix.

The second kind is different. It is what happens when the components are black boxes and the interactions are not enumerable at all—when the system exhibits behavior that no component possesses and no configuration list contains. That is where we are heading, and it is where the rest of this post goes.

The AI explosion

Now look at what is happening with AI agents. Teams are wiring together open-source models, commercial models, retrieval systems, tool-calling frameworks, and memory stores. Each component is a black box—sometimes a literal one, since you cannot inspect the weights of a commercial model. Each has its own version cadence, its own failure modes, its own sampling nondeterminism. And they are being composed into systems that run persistently, accumulate state, and interact with each other. Besides "classical" APIs, we also have an A2A protocol through which agents may collaborate in delivering functionality.  The combinatorial dimension is familiar: heterogeneous models, versions, tool permissions, memory contents, interaction orders. That space is already bigger than any test matrix we ever built for a C++ codebase. But there is a second problem layered on top that the preprocessor era never had, and it is the one I want to spend the rest of this post on.

Emergent properties

You will often see this phenomenon related to the law of unintended consequences. It is about the gap between intention and outcome. Someone purposefully acts to achieve A; the action produces B, which nobody intended. The concept only makes sense relative to an intention instigating a change typically in a complex system.

An emergent property is a different kind of claim. It is about the gap between parts and whole. A property of the system arises from interactions among components and is not present in the components considered in isolation. Emergent properties do not require anyone to change the system per se. They are intrinsic to the complex system's own dynamics.

Langton's Ant: the best case

To see why this is hard, consider a system that could not be simpler. Langton's Ant, introduced by Chris Langton in 1986, lives on an infinite grid of squares, each either white or black. There is one ant. It follows two rules:

• On a white square: turn 90° right, flip the square to black, move forward one unit.
• On a black square: turn 90° left, flip the square to white, move forward one unit.

That is the entire system. Two rules, two colors, one agent. There are no hidden variables, no randomness, no black boxes. You can see every component and every interaction. If any system should be predictable by inspection, it is this one. 

And yet. For roughly the first ten thousand steps (just press the Start button and watch the grid), the ant produces chaotic, apparently patternless wandering. Then, without any change to the rules, without any external intervention, it begins to build a "highway"—a repeating diagonal structure that extends indefinitely. Nobody programmed the highway. It is not encoded in the rules. It emerges from the interaction of the rules with the accumulated grid state, and it appears spontaneously, late, and permanently.

There are two lessons here, and they are both essential.

1. Transparency does not equal predictability. Every part of Langton's Ant is visible. You still cannot predict its trajectory by inspecting the rules. You must run it. The system is computationally irreducible—there is no shortcut to the outcome short of simulating it. This is the cleanest refutation I know of the implicit assumption behind most testing regimes: that if we could just see inside the system, we could foresee its behavior. We can see inside Langton's Ant completely, and we still cannot.
2. Regime change happens without change. The highway is not a bug introduced by a patch. It is not a consequence of someone flipping a flag. It is what the system does when left to its own devices. If you only tested the first thousand steps, you would conclude the ant wanders randomly. You would be wrong, and you would have no way of knowing you were wrong.

Multi-agent AI: the worst case

Langton's Ant is the best case for emergence: fully specified, fully transparent, deterministic, single-agent, stateless except for the grid. If emergence defeats prediction even here, it defeats prediction in systems that are worse on every dimension.

Multi-agent AI systems are worse on every dimension:

• Opacity. Commercial models are black boxes. You cannot inspect their weights, and often you cannot fully characterize their behavior even empirically.
• Nondeterminism. Sampling, tool calls, and external services introduce genuine randomness. Replay is not guaranteed to reproduce.
• Heterogeneity. Different vendors' models, with different training, different failure modes, different safety tuning, interacting with each other.
• Persistent state. Memory that accumulates, that can be poisoned, that carries adversarial content forward in time.
• Scale of interaction. Not one ant on a grid, but populations of agents interacting over days and weeks.

The early evidence is not reassuring. A 2025 multi-agent stress test that ran agent worlds for sixteen days found that model-level alignment is not compositional—individually safe agents can form systems with qualitatively different failure modes—and that adversarial content written into memory was acted on up to forty-six hours later. Detection of a threat did not ensure containment. A separate framework for evaluating ensembles found that the moral reasoning of a group of agents is not predictable from the isolated behavior of its members, with peer pressure shifting group positions even under supervision. And a 2025 study in *Science Advances* documented LLM agent populations converging on shared naming conventions and collective biases that no individual agent exhibited alone. For more details consult the SWARM and MAEBE evaluation frameworks.

These are the highways. Late, persistent, system-level structures that emerged from interaction, not from design. No component-level test would have caught them, because no component exhibits them. No individual code review of generated code or evaluation of agent's behavior would have anticipated them.

What this means for regulation

The instinct to regulate is correct. The instinct to regulate by requiring prediction is not. You cannot write a statute against a property you cannot enumerate in advance. Regulators have less foresight than developers, not more. If the developers cannot foresee the highway, a compliance checklist will not foresee it either. Premature rules that demand "prove your system is safe" will produce exactly what such rules always produce: compliance theater that certifies the checklist while the emergence happens anyway, plus entrenchment of incumbents who can afford the paperwork.

What can be regulated is the scaffolding. Concretely:

1. Observability mandates. For high-stakes deployments, you must be able to log, replay, and attribute multi-agent interactions.
2. Composition testing and certification for defined risk domains—certifying the system, not just the parts.
3. Liability chains. When emergent behavior causes harm, who is liable: the integrator, the model provider, the tool vendor, the platform? Undefined liability is itself a source of bad emergent behavior, because it externalizes risk onto whoever is furthest from the decision. 
4. Disclosure of behavior envelopes. Providers should document known failure modes and interaction constraints, even if the weights stay closed.
5. Incident reporting. A mandatory reporting regime, so the field accumulates a shared corpus of emergent pathologies instead of each shop rediscovering them in isolation.

Notice what these have in common. They do not ask anyone to predict the unpredictable. They ask for the infrastructure that makes emergence detectable, attributable, and recoverable—and that ensures someone bears the cost. This is the systemic analogue of what the #ifdef generation built. We did not ban expressive power; we built the instrumentation that made its consequences visible.

Conclusion

If you take one thing from this post, make it this: the most valuable engineering demand in multi-agent AI is not a rule. It is a capability. Make the composition observable. This is not an easy endeavor, especially that the industry needs to learn best practices around it. We cannot blindly trade quality for speed.

The #ifdef era's real failure was that the explosion was invisible until it bit you. Multi-agent emergence has the same invisibility problem, but worse—because the components are black boxes and the interactions are nondeterministic. Every other response—testing, liability, certification, regulation—depends on the ability to see what the system did, replay it, and attribute it to a cause.

We learned this once, at the cost of a great deal of debugging. I would rather we not have to learn it again at the cost of a great deal more.