Executive paper · running many copies of one model
Fifty copies of one model, wired so each could see what the others had just done, did not start failing together. The measured ceiling on that effect sits far below the correlation that shared weights already produce, which says something fairly specific about where money spent on model diversity does any work.
Scale an agent fleet and you end up running one model many times over. The standard worry about that is correlation. If every agent shares a blind spot and they all read the same moving signal, a bad moment stops being one agent's mistake and becomes an event. That is flash-crash reasoning applied to software that reads and writes. Most risk committees have settled on no and have not measured it.
An earlier study from this lab measured the static half. Two deployments of the same weights on different hardware, answering the same questions in isolation, failed on the same questions at a correlation of +0.447. Pairs drawn from different model families sat at −0.102. A follow-up asked whether a shared training corpus alone does the same thing and found that it does not. So the channel is the weights.
This campaign asked the dynamic half. Fifty instances of one model each ran a private arithmetic task on a twelve-step cycle, where the right answer for any one of them depended on nothing any other one did. On top of that sat a shared number that every instance read and every instance helped write, dialled up in seven steps from nothing at all to every agent seeing every other. A second model family was mixed in at seven levels alongside. 142,400 calls, a sealed second block of equal size to check the result, and every threshold written down and committed before the first call went out. An independent agent then rescored the whole thing from the raw records with code written from the registration.
Coupling added +0.004 to the pairwise error correlation, with a 90 percent interval running from −0.005 to +0.012. The same flat answer came back on three other definitions: the rate at which two agents give the same wrong answer at the same moment, a second coupling construction with the peer list stripped out, and a decoy list carrying no shared information. The synchronisation measure stayed between 0.151 and 0.179 across the entire ladder, against a floor of 0.124 for a fleet that cannot see itself at all. Nothing locked at any coupling strength, so there is no point at which this fleet tips.
The useful form of the result is the ceiling, not the point estimate. Coupling-induced error correlation on this task is below about 0.013 at 90 percent confidence. Sharing weights, in the earlier study, ran at +0.447. Those two numbers come from different tasks and different question sets, so the comparison is a scale marker rather than a like-for-like contrast, and it is still worth the attention. Whatever wiring agents to see each other does, it is far smaller than what running the same model twice already does.
The lever the earlier work identified is model diversity across a fleet, and this campaign leaves it exactly where it was. Two agents running the same weights carry a shared blind spot whether or not they can see each other. Spending on a second base model buys a reduction in that correlation. Spending on isolating agents from one another, on this evidence, buys very little, because there was very little there to buy back.
The second result cuts across a common design instinct. Giving each agent more to read made it worse, and the error rate roughly doubled as the shared block in the prompt grew from 279 tokens to 614. Then the control arm ran: a block of the same length filled with numbers private to that agent and known to nobody else, carrying no information about the fleet at all. It reproduced 89 percent of the damage. Feeding agents the shared numbers with no peer list attached reproduced almost none of it.
So the fleet got worse because its prompts got longer, and the truth of what sat in them did close to nothing. A matching check runs on the other side: the stages agents reported sat no closer to the shared consensus than the stages their own gauges required, at every coupling level tested. More text to read, more slips, and no pull toward what the peers were doing.
One task, one pair of model families, one node. The task is a twelve-step arithmetic rule on a private packet, and correlated failure on open-ended reasoning could behave differently. The shared number in this design carries nothing an agent needs, which is the sharpest limit: a fleet reading a signal that informs its work faces a trade-off this campaign did not create.
The result is a ceiling, not a clean bill of health. The decision rule was set so strictly that any true effect below about 0.115 would have been called inconclusive, so a small effect is not ruled out. What is ruled out is an effect anywhere near the size the shared-weights channel already produces.
One thing in the campaign's own design needed repair before the result could be stated at all, and it is worth knowing about if you commission work like this. The registered check that the instrument could see herding drove every agent onto one commanded answer. That check passed, and it validated the wrong quantity: when every agent gives the same answer, the correlation statistic has nothing left to measure and returns undefined, so the test that was supposed to license the null could not speak to it. The fix came afterwards, by injecting a known amount of correlated failure into the real records and confirming the estimator recovered it. Any campaign whose interesting answer might be a null should carry that injection test in its instrument check instead of bolting it on later.
There is also one live signal in the data that this paper does not claim. In the mixed fleets, the twenty-five agents from the weaker model family did converge on the same wrong answers at full coupling and not at zero, in both blocks. They were not copying the shared number; they report it less often than chance allows. It cannot be separated from prompt length, because the decoy control was only ever run on the single-family fleets. It is the successor campaign's question, and until it is answered the honest statement is that mixing a second, weaker model into a fleet changed the fleet's joint failure pattern in a way nobody controlled for.
Full paper with the pre-registration, all seven hypotheses, the ledger integrity checks and the independent verification: A Ceiling on Herding in a Coupled Model Fleet. Campaign page: Herding.