Multi-agent LLMs can't explore each other: here's the data

Q: What does "multi-agent LLMs fail to explore" mean?

It means LLM agents don't try different collaborators when working in a group. They pick one early, lock onto it, and never query the others, even when they picked the wrong one.

Q: What is UCB and how does it help?

UCB (Upper Confidence Bound) is an algorithm that adds an uncertainty bonus to each option's score, forcing exploration of less-tested options. The bonus shrinks automatically as evidence accumulates, creating a principled exploration-exploitation balance.

Q: Who should not worry about this finding?

Teams using predefined communication graphs (A talks to B, B talks to C) or systems with a single orchestrator that routes tasks deterministically. The finding matters most when agents are free to choose their own collaborators.

New research shows LLM agents lock onto the same collaborator 294 out of 297 times. A two-armed bandit reveals why multi-agent systems need algorithms, not prompts, for exploration.

TL;DR: LLM agents in multi-agent systems don’t explore their peers. They commit to one collaborator in the first few rounds and never reconsider, even when they picked the wrong one. GPT-5 picks the same peer 294 out of 297 times. Prompting cannot fix this. In-context exploration performs worse than random selection. The fix is algorithmic: a 30-line UCB bandit layer that overrides peer selection. New research from UW-Madison proves this across contextual and parametric diversity settings.

Key takeaways:

LLM agents produce bimodal peer-selection distributions: they pick one collaborator either 0 or 50 times out of 50, never in between

In-context exploration (prompting agents to explore) underperforms random peer selection. The LLM’s own judgment is worse than a die roll

GPT-5 is the worst offender: 294 out of 297 interactions went to the same Qwen-7B agent, ignoring the other two in the pool

The fix is a contextual bandit layer that tracks per-peer performance with an uncertainty bonus. No prompts needed

The more diverse your agent pool, the more you lose by not exploring: regret grows linearly with capability diversity

How do multi-agent systems decide who to talk to?

Every multi-agent framework solves the same problem: you have several agents, each with different context, tools, or capabilities, and they need to exchange information to complete a task.

CrewAI lets agents delegate to each other. LangGraph builds state machines where agents route work dynamically. AutoGen turns conversations between agents into programmable workflows. ChatDev assigns roles and lets them collaborate.

All of them assume one thing: given enough information and a clear prompt, an agent can figure out who to talk to.

That assumption is wrong.

This paper is from University of Wisconsin-Madison and UC Santa Barbara, published on arXiv in July 2026 (arxiv.org/abs/2607.11250). Code and data are available at github.com/deeplearning-wisc/mace.

Here is the claim this paper proves: LLM agents cannot explore their peers effectively. They commit prematurely to one collaborator, ignore the rest, and never reconsider. Even when the data clearly shows they picked the wrong one.

It is not a small-model problem. It is not a prompting problem. It is a structural limitation of how LLMs approach exploration in multi-agent settings.

What happens when you ask an LLM to choose between two peers?

The paper starts with the simplest possible test. A two-armed bandit. The researchers tested Qwen2.5-7B, GPT-4, and GPT-5 on this setup (arxiv 2607.11250).

You are an LLM agent. There are two peers, A and B. One answers correctly 60% of the time. The other answers correctly 50% of the time. You do not know which is which. You get 50 rounds. Each round, you pick a peer, they answer, and you see if they got it right.

The optimal strategy: try both a few times, identify the better one, then exploit it. That is what a classical algorithm (UCB) does. Its distribution of peer selections is a clean normal curve centered on the better peer.

The LLM agents do something completely different.

Every LLM tested produces a bimodal distribution: Qwen2.5-7B, GPT-4, and GPT-5 all show the same pattern. The histogram has mass at the extremes: either 0 selections of the better peer or all 50. There is no middle ground. No gradual convergence. The agent picks a favourite in rounds 1-3 and locks onto it for the remaining 47 rounds.

When the agent locked onto the wrong peer, it stayed there for all 50 rounds. The data was visible every round. The inferior peer’s success rate was lower. It did not matter. The commitment was made.

The catch: this is not a fluke of a particular model. GPT-5, the most capable model in the test, showed the most rigid lock-in. In the parametric diversity experiment, with four agents using different model backends, GPT-5 selected the same Qwen-7B agent in 294 out of 297 interactions. It almost never queried the other two.

Can you prompt LLMs to explore more?

This is the natural question. If agents lock on because they are not instructed to explore, add exploration instructions.

The paper tested this extensively. The in-context exploration baseline receives the full interaction history: how many times each peer was queried, how often they answered correctly, plus an explicit prompt to balance exploration and exploitation.

It has access to the same information as the algorithmic solution. It relies on the LLM’s reasoning instead of a structured decision rule.

The result: in-context exploration underperforms random peer selection. Not matches it. Underperforms.

Under contextual diversity (where each agent has different information), autonomous LLM agents making their own peer choices are less reliable than a naive stochastic policy that picks uniformly at random. The LLM’s own strategic reasoning about who to talk to is actively worse than throwing a die.

Why it works: the mechanism that makes LLMs decisive in single-agent settings, commit to a direction and follow through, becomes a liability in multi-agent settings. The same confidence that helps a single agent complete a task prevents it from reconsidering its collaborator choices.

Why does UCB solve the problem that LLMs cannot?

UCB stands for Upper Confidence Bound. It is a bandit algorithm from 2002, and it solves the exploration problem with one idea: track uncertainty, not only success rate.

Here is how UCB works for peer selection:

Each peer gets a score: success_rate + uncertainty_bonus. The uncertainty bonus is sqrt(2 * ln(total_rounds) / times_queried). When you have queried a peer only a few times, the bonus is large. It forces you to try them. As evidence accumulates, the bonus shrinks. The transition from exploration to exploitation is automatic, driven by the data, not a schedule.

This is the piece the LLM cannot replicate. The LLM has no principled way of answering “how many times should I try before I am sure?” It uses heuristics like “this one worked once, it must be the best.” Those heuristics fail in multi-agent settings.

UCB answers that question mathematically. Hoeffding’s inequality provides a bound: with high probability, the true success rate is within this margin of the observed rate. When the margin is large, explore. When it is small, exploit.

LinUCB extends this by replacing identity with context. Instead of tracking “how many times have I queried peer A,” it tracks “how many times have I queried peer A on math problems versus creative tasks.” The uncertainty bonus operates on a feature space, not raw counts. The agent can distinguish between “I have not tried peer A on this type of task” and “I have not tried peer A at all.”

The paper’s MACE framework implements this with four features: response diversity (how different is the peer’s answer), peer distinctiveness (how unique is this agent), historical performance, and interaction round.

When does this finding not apply?

If your agent system uses a fixed communication graph, say A always talks to B, B to C, and C to A, the exploration problem never arises. The routing is deterministic. No agent needs to decide who to collaborate with.

If your system has a single orchestrator that assigns tasks based on rules, routing code review to the coding agent and research to the search agent, the decision is made by the system, not the agent.

This finding matters when agents have the freedom to choose their own collaborators. The frameworks that advertise “autonomous delegation” and “dynamic peer selection” are the ones creating this failure mode.

The other case: homogeneous agent pools. When all agents use the same model with the same capabilities, the cost of picking the wrong peer is low. The paper proves this formally: the exploration benefit scales with capability diversity (δ). When δ ≈ 0, exploration does not matter. When δ is large, non-exploring agents accumulate regret linearly.

What should you do if you are building multi-agent systems?

Two options, ordered by control.

Fixed communication graph. Define the routing upfront. If you are using CrewAI, disable autonomous delegation. If you are using LangGraph, set a static edge from each node to the next. The paper’s pre-defined baseline, where each agent talks to its right-side neighbor, consistently beats in-context exploration.

Bandit layer. Add a 30-line UCB module that tracks per-peer performance and overrides the LLM’s peer selection. The formula:

score = success_rate + sqrt(2 * ln(total_rounds) / times_queried)

Score each peer, pick the highest, update after each interaction. That is the whole algorithm. It is simpler than any prompt engineering workaround and empirically works better.

Do not add “explore different perspectives” to your system prompt. The paper tested it. It does not work. The LLM’s exploration reasoning is systematically worse than random. The fix is structural. Take peer selection out of the LLM’s hands.

FAQ

What does “multi-agent LLMs fail to explore” mean? It means LLM agents do not try different collaborators when working in a group. They pick one early, lock onto it, and never query the others, even when they picked the wrong one.

Does prompting agents to “explore more” fix the problem? No. The paper tested in-context exploration with full history and explicit exploration instructions. It still underperformed random peer selection. Prompting alone cannot fix this.

What is UCB and how does it help? UCB is an algorithm that adds an uncertainty bonus to each option’s score, forcing exploration of less-tested options. The bonus shrinks automatically as evidence accumulates, creating a principled exploration-exploitation balance.

Does this affect all LLMs or only small ones? All of them. GPT-5 showed the most rigid lock-in, picking the same peer 294 out of 297 times. The failure is not a capacity issue. It is a structural limitation of how LLMs approach exploration.

Who should not worry about this finding? Teams using predefined communication graphs or deterministic orchestrators do not face this problem. It matters most when agents are free to choose collaborators.

CrewAI vs LangGraph: which agent framework to use. A practical comparison of how different frameworks handle agent-to-agent communication.
The Proactive Agent Problem: What Claude Fable Got Right. Why agent autonomy is a feature when goals are clear and a liability when they are not.
Your context window is a workspace, not a filing cabinet. How context engineering shapes what agents remember and forget during collaboration.

This article was published on Agentic Up (https://agenticup.dev): practical guides for developers and founders building with AI agents. Reach\u00a0me\u00a0at\[email protected]