---
title: "The phase transition hiding in your agent's skill library"
canonical: "https://agenticup.dev/posts/skill-library-phase-transition-agents/"
pubDate: "2026-07-01T00:00:00.000Z"
description: "Skill-based single agents match multi-agent performance at 54% fewer tokens. But selection accuracy collapses beyond ~80-90 skills , a phase transition not gradual decay."
tags: [ai-agents, agent-architecture, research, multi-agent-systems]
---

**TL;DR:** UBC researchers formalized converting multi-agent systems into single agents with skill libraries, saving 54% tokens and 50% latency while matching accuracy. But there's a catch: skill selection accuracy stays stable up to a critical threshold (~80-90 skills), then collapses sharply . Not gradually. Semantic confusability among similar skills drives the collapse, not library size alone. Hierarchical routing restores accuracy from ~45% to ~85% at 120+ skills.

> **Key takeaways:**
> - MAS → SAS compilation saves 54% tokens and 50% latency while matching accuracy on GSM8K, HumanEval, and HotpotQA
> - Selection accuracy follows a phase transition: stable up to ~80-90 skills, then drops to ~20% at 200 skills
> - Semantic confusability, not library size, drives the collapse. Unique skills stay at 100% accuracy
> - Hierarchical routing (coarse-to-fine) restores ~72-85% accuracy at 120+ skills
> - Keep flat skill libraries under ~50 skills, audit overlap, adopt hierarchy beyond that

---

Here's a puzzle I keep running into.

Every team I talk to is building multi-agent systems. They have a writer agent, a reviewer agent, a coder agent, a debugger agent, a planner agent. The architecture diagrams look like network topologies. Token costs look like AWS bills.

The natural alternative, one agent with a library of skills, gets dismissed as a toy. It shouldn't be.

## What the paper found

Xiaoxiao Li at UBC (CIFAR AI Chair) formalized something I've felt but couldn't prove. A multi-agent system can be compiled into a single agent with skills by encoding each agent's behavior as a skill descriptor and execution policy. The math works out: pipeline, router-workers, and iterative refinement architectures are all "compilable."

The results on three benchmarks:

| Metric | MAS | SAS (compiled) | Change |
|-------:|:---:|:--------------:|:------:|
| GSM8K accuracy | 94.0% | 92.0% | -2.0% |
| HumanEval accuracy | 100.0% | 100.0% | 0.0% |
| HotpotQA accuracy | 84.0% | 88.0% | +4.0% |
| Avg. tokens | ~2,389 | ~1,060 | **-53.7%** |
| Avg. latency | ~9,821ms | ~5,022ms | **-49.5%** |
| API calls | 3-4 | 1 | **-75%** |

The SAS uses one API call. One context window. No inter-agent handoff overhead.

That alone is useful. But the paper's real contribution is the question it asks next: *what happens when the skill library grows?*

## The phase transition

Li tested selection accuracy across libraries from 5 to 200 skills, controlling for semantic similarity. The pattern is not a gentle slope. It's a cliff.

At |S| ≤ 20, accuracy stays above 90%. Between 30-75 skills, it holds steady. At 80-90 skills, it starts dropping. At 200 skills, accuracy is about 20%.

The fitted scaling law:

ACC ≈ α / (1 + (|S|/κ)^γ)

Where κ ≈ 85-92 (the capacity threshold where accuracy halves) and γ > 1.5 (super-linear decay (the phase transition)).

The cognitive science parallel is deliberate. Hick's Law says human reaction time scales logarithmically with choice count. It breaks down beyond ~8 options. Li shows LLM skill selection has an analogous capacity limit, with a phase transition instead of a log curve.

## What actually breaks selection

Here's the part that matters for builders. Library size alone isn't the problem. Confusability is.

In a controlled experiment, Li created skill libraries with three conditions:
- **No competitors:** Each skill is semantically unique
- **Low confusability:** 1 semantically similar "competitor" per skill
- **High confusability:** 2 competitors per skill

At |S| = 20, the no-competitor condition scored 100% accuracy. Adding one competitor dropped it 7-30%. Adding two dropped it 17-63%.

This means: if you have 10 well-separated skills, selection is near-perfect. If those 10 skills include "summarize text," "summarize code," "summarize meeting notes," and "summarize email," you're in the danger zone.

Skill descriptors matter. "Process data" is not a skill name. "Compute rolling 7-day average across time-series metrics" — that's a skill.

## The fix: hierarchy

Flat selection degrades because the model must scan all options at once. Hierarchical routing breaks the decision into two stages:

1. Pick a cluster (10-40 categories, below the capacity threshold)
2. Pick a skill within that cluster (2-3 options, trivial by comparison)

At |S| = 120, flat accuracy is ~45-63%. Hierarchical accuracy is ~72-85%. For GPT-4o-mini, the improvement is +37-40% absolute.

The paper tested two hierarchies: naive domain grouping and confusability-aware grouping. Both worked similarly when domain boundaries happened to align with confusability clusters. The key design rule: first-stage categories must be semantically distinct. Put confusable skills together in the second stage where the small pool makes selection easy.

## When NOT to use this

Three conditions make compilation fail:

- **Parallel sampling:** Independent agents running in parallel, best-of-n selection
- **Private information:** Agents with hidden state
- **Adversarial objectives:** Debate, opponent modeling

These require true parallelism or information asymmetry that a single context window can't replicate. For everything else (pipelines, router-workers, iterative refinement), compilation is faithful.

## The practical playbook

1. **Keep flat libraries under ~50 skills.** Above that, accuracy degrades non-linearly.
2. **Audit semantic overlap before adding skills.** If two skills sound similar, merge or differentiate them.
3. **Adopt hierarchy at scale.** Group by domain, keep first-stage categories distinct, keep second-stage pools tiny.
4. **Write specific descriptors.** "Compute sum of values" beats "process data."
5. **Match model to library size.** Stronger models have slightly higher capacity thresholds, but hierarchy helps all models.

## What's still open

This is a UBC technical report, not a peer-reviewed paper. The experiments use synthetic skill libraries, not real-world agent skills from production systems. The model coverage is limited to GPT-4o-mini and GPT-4o. End-to-end task accuracy (not just selection accuracy) needs more study.

But the phase transition finding is robust: R² > 0.97 on the scaling law fit, consistent across two models, and grounded in established cognitive theory.

The implication for builders: if your agent has more than 50 skills and you're using flat selection, you've already hit the phase transition. You just haven't measured it yet.

## FAQ

> **What is a skill in this context?**
> A skill is a schema-bounded operation with a semantic descriptor, an input-output signature, and an execution policy. It's like a tool, but instead of being automatically triggered, the model must choose it based on semantic match.
>
> **Does this mean multi-agent systems are obsolete?**
> No. Many production agent systems need parallel execution, private state, or heterogeneous models. All of those fail the compilability conditions. But for sequential pipeline architectures, a compiled SAS matches performance at half the cost.
>
> **How do I know if my skills are too similar?**
> If you can't describe each skill in one sentence that distinguishes it from all others, they're too similar.
>
> **Is this the same as Anthropic's agent skills?**
> The paper builds on Anthropic's concept of agent skills and formalizes the scaling behavior that Anthropic's documentation doesn't cover.

## Related Posts

- [When better models aren't better agents](/posts/when-better-model-isnt-better-agent/). The scaling of agent capability doesn't follow model capability curves either.
- [Code as agent harness: a survey of 7 production systems](/posts/code-as-agent-harness-survey/). Different approaches to agent architecture, including single-agent and multi-agent patterns.
- [Why agents break in production](/posts/agents-break-in-production/). The failure modes that matter when you ship an agent system.

---

This article was published on Agentic Up (https://agenticup.dev): practical guides for developers and founders building with AI agents. Reach me at hello@agenticup.dev