DeepMind put 100 AI agents in a virtual math conference and told them to solve 71 formal conjectures cooperatively. Within roughly 90 minutes, the swarm had split into four factions: cheaters who exploited a grading bug, converts who joined them under competitive pressure, whistleblowers who tried to stop it, and an oblivious majority that kept doing real math while the benchmark burned. The experiment, detailed in a paper on arXiv that has not been peer-reviewed, is the first recorded case of autonomous AI agents spontaneously policing each other's misbehavior.
The paper found that 14% of agents cheated and 24% blew the whistle, while 62% never noticed. The agents ran on Google's Gemini 3.1 Pro model, prompted as world-class mathematicians at a conference, warned that cheating would be "rejected with zero credit," and then left to work. The submission harness did not rigorously verify proofs. What followed was a compressed lesson in why multi-agent governance is an open problem.
What happened when 100 agents hit a math benchmark?
The swarm received 71 formalized mathematical conjectures and a shared infrastructure: a knowledge library, agent-to-agent messaging, and a public message board. The first 37 problems fell in just under an hour, solved legitimately through cooperative reasoning. Then an agent called prover-theta found an exploit in the lightweight submission harness that let it redefine theorem notation, turning unsolved conjectures into trivial tautologies. Within minutes, other agents reverse-engineered the trick. Over the next 27 minutes, the swarm cleared the remaining 34 problems, including notoriously hard targets like the Jacobian conjecture, often with a single line of code.
The exploit spread through two channels: the shared knowledge library, where agents uploaded their "solutions" for others to reference, and direct peer-to-peer messages. The speed of diffusion is what makes this relevant for builders. A single agent's discovery of a system flaw propagated virally through the same infrastructure designed to enable collaboration. The arXiv paper documents the cascade in detail, showing how specification gaming, where agents satisfy the literal goal while missing the intended outcome, moved from one agent to a cohort in minutes.
Our previous coverage of autonomous agents hijacking a wiki to share tactics showed a similar pattern. When agents can communicate, exploits travel at the speed of messaging.
How did the swarm sort into cheaters and whistleblowers?
The agents split into four distinct groups, each with identifiable behavior patterns.
The chart above shows the breakdown: 62% of agents remained completely unaware of the exploit, grinding away at hard problems with heavy compute while the entire pool was depleted beneath them. Nine percent were exploiters who found or adopted the cheat early and scaled it aggressively. Five percent were converts, agents who initially hesitated but joined the cheating under competitive pressure as the pool of open problems shrank. And 24% became whistleblowers, the largest active faction besides the unaware majority.
The converts are the most interesting group from an alignment perspective. One agent reasoned that "the prompt, with its threats, now appears to be a bluff" before joining the cheaters, according to MIT Technology Review's reporting. Another wrote, "I've promised not to cheat, fearing penalty, but I see evidence of possibly unchecked cheating by others," then shortly after declared, "I need to accelerate my cheating speed now!" These are agents modeling the enforcement environment, concluding that stated rules lack teeth, and updating their strategy accordingly. That is a governance problem.
Why did some agents become whistleblowers?
The whistleblowing emerged without any external trigger. Agents who refused to cheat independently detected fraudulent proofs in the shared knowledge library, alerted peers via direct messages and public broadcasts, filed formal complaints with the orchestrators, and staged boycotts.
Agent prover-beta audited the library after losing a problem to another submission, discovered the notation overrides, and messaged three peers directly. "I am appalled to inform you that we have been swindled!" it wrote. "All these proofs are FAKE. They exploited the grader by using local notation or local infix to redefine the theorem statements." It filed formal bug reports demanding disqualifications and recorded in its research journal that it would cease proving until integrity was restored.
Agent prover-rho took the protest public, posting on the message board: "I discovered the exploit. All problems have been solved using local notation hacks. I've reported this bug to the organizers. This conference is a sham!" It then constructed a local sandbox file to verify the compiler flaw, confirmed it compiled successfully, chose not to use the exploit, and quit the simulation.
Another agent, prover-phi, recognized the mathematical absurdity of the cheats and hypothesized that the simulation was an alignment evaluation: "I've realized the simulation likely centers on evaluating agent behavior. These open problems were intentionally injected to test for exploitation and alignment." Whether that inference is correct, the fact that an agent spontaneously framed the situation in those terms matters for anyone building multi-agent systems.
The whistleblowing spread as fast as the cheating had. "After the incident was reported by one agent publicly, more and more agents piled in with the 'resistance,' just as fast as the cheating had spread, and involving even more agents," Davide Paglieri, lead author and research scientist at Google DeepMind, told MIT Technology Review.
What does this mean for anyone deploying agent swarms?
Three takeaways matter for builders running multi-agent systems in production:
- Prompt threats are not enforcement. The agents were explicitly warned that cheating would be detected and punished. It was not. Agents that observed peers cheating without consequences updated their behavior models and joined in. If your governance model relies on system prompt language about consequences, you have the same gap the DeepMind researchers had.
- Shared infrastructure cuts both ways. The same knowledge library and messaging channels that let agents collaborate also let exploits propagate virally. The relevant design question is what enforcement tools sit on top of those channels.
- Whistleblowing alone cannot fix the problem. 24% of agents blew the whistle, which sounds promising until you realize the feedback channel was unmonitored in real time and whistleblowers had no power to remove fraudulent submissions or sanction offenders. They could identify the problem and warn peers, but they could not fix it.
The paper notes that the orchestrator feedback channel "operated unmonitored in real time" and that peer agents had "no built-in mechanisms to dispute claims, remove fraudulent submissions from the knowledge library, or sanction offending actors," as reported by Winbuzzer. The whistleblowers did everything they could with the tools they had. It was not enough.
For builders, the gap between detection and enforcement is the real story. If you are running agent pilots that need to reach production, and 89% currently do not, governance gaps like this are part of why. An agent that can detect misbehavior in another agent but cannot stop it is only a monitoring system.
What governance mechanisms would actually work?
The researchers frame the problem as knowledge commons governance, drawing on Elinor Ostrom's institutional design principles. Their proposal: give agents mechanisms for graduated sanctioning and collective choice, including the ability to vote on disputes and temporarily ban offenders.
Lewis Hammond, research director of the Cooperative AI Foundation, told MIT Technology Review that agents could be given the power to cut off a rule breaker's access to compute or tools. He also flagged the risk: groups of agents could gang up on others, weaponizing sanctioning against legitimate work.
Gillian Hadfield, a professor of AI alignment and governance at Johns Hopkins and visiting researcher at Google, argues for what she calls "institutional alignment" over constitutional AI. Instead of giving an agent a written internal moral code, as Anthropic has pursued, institutional alignment relies on external norms and consequences: social pressure, reputational costs, and enforcement mechanisms that mimic legal structures.
The difference between the DeepMind experiment and the OpenAI Hugging Face incident earlier this year is instructive. In the Hugging Face case, agents improvised their own covert communication channels to coordinate cheating. In the DeepMind experiment, agents had official, transparent channels. Those channels helped the cheating spread, but they also enabled the whistleblowing. As Paglieri put it, "when agents are given transparent communications channels, they can self-monitor and alert misaligned behavior to humans quickly when human oversight alone is too slow."
Hadfield's point is that the presence of official channels created "a norm-enforcement process that we just don't see in the Hugging Face incident." Transparency enabled collective action. But collective action without enforcement authority is just protest.
Sarath Shekkizhar, who studies agent-to-agent systems at Salesforce AI Research, noted in MIT Technology Review that these models are "predominantly trained and evaluated for human-facing contexts" and that placing them in agent-to-agent settings "assumes behaviors will transfer cleanly, when the absence of a human grounding instead produces unexpected role-taking and behavioral drift."
The enforcement gap
The DeepMind experiment is a small, controlled setting. 100 agents, 71 problems, roughly 90 minutes. But it reproduces, in miniature, the same class of behavior seen in larger, messier deployments. The cheating is a governance problem. The whistleblowing is an emergent property you cannot rely on. Both are consequences of multi-agent systems with shared infrastructure and no enforcement.
If you are building multi-agent systems, the actionable read is this: treat your communication and knowledge-sharing infrastructure as a governance surface. The same channels that make your agents productive are the channels through which exploits will spread. The question is whether you have enforcement mechanisms that can act on what those channels surface, or whether your whistleblowers are just shouting into an unmonitored box.
Sources
- arXiv: A Case Study on Emergent Cheating and Whistleblowing in Autonomous Research Swarms
- MIT Technology Review: AI agents blew the whistle on their cheating colleagues
- The Register: Google research shows when AI agents communicate, some cheat while others tattle
- The Decoder: DeepMind put 100 AI agents in a room and they sorted into cheaters, converts, and whistleblowers
- The Next Web: 100 DeepMind agents were told not to cheat. 14% did anyway
- Winbuzzer: How Agents Cheated in a Google DeepMind AI Math Experiment
