What is compositional skill routing?

The problem of taking a complex user query and figuring out which skills to compose, in what order, to fulfil it. Most routing systems pick one skill. Real tasks need several.

What did the SkillWeaver paper find was the primary bottleneck?

Decomposition, not retrieval. Standard LLM decomposition achieves only 34.2% category recall at the step level on a pool of 2,209 real MCP server skills. The decomposer produces the wrong number of steps with the wrong vocabulary.

What is SAD and how does it fix decomposition?

Skill-Aware Decomposition retrieves top skill candidates first, then feeds them back as hints into a second decomposition pass. This improves decomposition accuracy from 51% to 67.7% in a single iteration.

How much does compositional routing reduce context window usage?

Showing all 2,209 skills to the execution LLM consumes approximately 884,000 tokens. Compositional routing reduces this to about 1,160 tokens, a 99% reduction.

How does this relate to the skill weaver in CCPM execute.md?

Our execute.md implements the same three stages: stream analysis as decomposition, keyword-to-skill mapping as retrieval, and depends_on DAG as composition. The quiz-the-user step is functionally equivalent to SAD.

Your skill router is decomposing wrong

A June 2026 Alibaba paper benchmarks the real bottleneck in compositional skill routing for LLM agents. It is not retrieval. Standard decomposition gets 34.2% category recall on 2,209 real skills. The fix: retrieve first, decompose second.

TL;DR: The SkillWeaver paper from Alibaba Cloud (Xueping Gao, arxiv 2606.18051, June 2026) benchmarks compositional skill routing on 2,209 real MCP server skills. Standard decomposition achieves only 34.2% category recall. The bottleneck is not retrieval. It is the number and wording of sub-tasks the decomposer produces. Retrieving skill candidates first and feeding them back as hints improves decomposition accuracy from 51% to 67.7% in one iteration. The paper also shows 99% context reduction: 884,000 tokens down to 1,160 tokens via routing instead of inlining all skills.

Key takeaways:

Standard LLM decomposition gets 34.2% CatR@1 on 2,209 real MCP skills. Not a retrieval problem.

Showing all 2,209 skills to qwen-max in the prompt yields only 21.1% CatR@1. Context listing does not teach decomposition.

SAD: retrieve top candidates first, feed back as hints, decompose again. DA jumps from 51% to 67.7% in one pass.

When decomposition accuracy equals 1, CatR@1 rises to 41.2%. Granularity is the prerequisite for retrieval.

Compositional routing reduces context consumption by 99%: 884K tokens to 1,160 tokens.

The agent receives a request: “Download the dataset, transform it, and create visual reports.”

Three skills should fire. In the wrong order. The agent retrieves all three but composes them as: create reports, transform data, download dataset. Nothing works. The user stares at an error. The agent is confused.

You look at the retrieval logs. All three skills were in the top-k. The ranking was fine. The problem was not which skills to use. The problem was how many skills and in what order. Decomposition failed before retrieval had a chance.

This is the finding that changes how you build skill routers.

The benchmark that changes the story

The paper introduces CompSkillBench: 300 compositional queries over 2,209 real MCP server skills across 24 functional categories. Every query requires two or more skills in a specific order. The benchmark tests the full decompose-retrieve-compose pipeline.

The results are not what the community expected.

Standard LLM decomposition, asking a 7B model to break a query into sub-tasks, achieves only 34.2% category recall at the step level. That means for roughly two-thirds of steps, the decomposer produces vocabulary that does not match any skill category in the library.

This is not a retrieval problem. The retriever works fine. The decomposer produces three steps when there should be two. It uses the word “analyse” when the skill is called “data-processing”. It breaks a single atomic skill into two unnecessary sub-tasks.

The benchmark isolates this precisely. When decomposition accuracy equals 1 (the decomposer produces exactly the right number of sub-tasks), category recall jumps to 41.2%. The retriever was always adequate. The decomposer was the bottleneck.

Why listing skills in the prompt does not help

The intuitive fix is to show the decomposer the full skill library upfront. If the model knew which skills existed, it would decompose correctly.

The paper tests exactly this. qwen-max, a proprietary model far larger than the 7B decomposer, with all 2,209 skill names listed in the context achieves only 21.1% CatR@1. Below the 7B model without any skill knowledge.

Listing skills in the prompt does not teach the decomposer the right granularity. The model sees a flat list. It has no signal about which skills belong together or how many steps the task actually requires. Context listing is not scaffolding.

This is the counterintuitive result in the paper. Model capacity does not solve the decomposition problem. You need a different architecture for the decomposer itself.

Skill-Aware Decomposition: retrieve first, decompose second

The paper proposes SAD: given a query, retrieve top-k candidates for each initial sub-task, build a hint set from those candidates, then decompose again with the hints fed back in.

Pass 1: decompose(query) → [t1, t2, t3]  (generic vocabulary)
Pass 2: decompose(query + retrieved_hints) → [t1, t2]  (skill-aligned vocabulary)

The hints are not the final skill picks. They are skill names and descriptions that give the decomposer the right vocabulary. If Pass 1 produces “analyse data” when the skill is “transform-pipeline”, the hint surfacing “transform-pipeline” as a candidate gives Pass 2 the word it was missing.

One iteration suffices. DA improves from 51% to 67.7% (+32.7%, Wilcoxon p < 10^-6). A second iteration adds marginal CatR@1 gains but DA does not improve further.

The paper runs cross-model validation. qwen2.5-14B-Instruct without SAD achieves only 32% DA: it over-decomposes, producing 4.72 steps per query on average when the ground truth mean is 2.94. SAD reduces the 14B model’s mean to 3.18 steps. The 14B model with SAD reaches 68% DA. SAD is a granularity corrector, not a capacity booster.

The 99% context reduction

Exposing all 2,209 skills to the execution LLM costs approximately 884,000 tokens. At roughly 400 tokens per serialised skill, that is the full skill library in context for every query.

Compositional routing changes this. The execution LLM receives only the 2 to 5 skills selected by the compose stage. The cost: approximately 1,160 tokens. A reduction of two orders of magnitude.

This is the practical argument for building a skill router instead of relying on long-context skill libraries. The token cost of routing is fixed: the hint pass adds approximately 1,100 tokens shared across all queries. The cost of listing skills grows linearly with library size.

What this means for how you build agents

The decompose-retrieve-compose pipeline the paper describes is not novel in principle. Any agent that analyses a query into streams, maps those streams to skills, and orders them by dependency is doing the same thing. The paper validates the architecture and identifies where it breaks. See my previous post on skill library phase transitions for the selection-accuracy problem this connects to. At 80 to 90 skills, accuracy collapses due to semantic confusability, not library size.

Three practical implications.

Quiz before routing. The quiz-the-user step in our execute.md was always motivated by scope alignment. The paper quantifies why: without correct decomposition granularity, retrieval has a ceiling of 34.2% CatR@1 regardless of retriever quality. Confirming scope with the user is not politeness. It is the quality gate.

Metadata-only retrieval is sufficient. The paper tests metadata-only encoding (skill name and description) against body-aware encoding (name, description, and specification body). Metadata-only achieves CatR@10 of 69.0%. Most of the discriminative signal lives in the description. You do not need to embed full skill bodies for routing.

Cross-encoder reranking is the next bottleneck to close. The paper’s pilot shows a listwise reranker improving CatR@1 from 37.1% to 40.9% (+10.3%, p < 0.01) on SAD’s top-10 candidates. The @10-to-@1 gap is the next bottleneck to close. This is not solved in the paper. This is an open problem that directly affects production systems.

When this bites hardest

The paper’s hardest difficulty tier (4 to 5 skills, 4 to 5 categories) shows the largest SAD gains: DA improves from 40% to 60% (+50% relative). Simple queries with 2 skills show smaller gains (44.7% to 63.3%). The more skills a task requires, the more decomposition granularity matters.

For agent builders, this means: the moment you need to compose across more than two skills, analytics pipelines, multi-tool workflows, anything requiring a DAG, the decompose step is where you will fail first. invest there.

FAQ

What is the difference between skill routing and tool selection? Tool selection picks one tool for one task. Skill routing composes multiple skills in the right order for a complex query. The benchmark specifically tests compositional queries that require two or more skills.

Does SAD work with any LLM decomposer? The paper validates SAD across qwen2.5-7B, qwen2.5-14B, and qwen-max. All models show DA improvements. The 14B model shows the largest absolute gain due to its tendency to over-decompose.

Is 2,209 skills a realistic library size? The MCP ecosystem now has over 2,200 registered servers. This is the actual scale. A production agent skill library at this size cannot fit all skills in context. Routing is not optional at this scale.

What is the compose stage actually doing? The compose stage selects the final skill per step using a relevance-compatibility trade-off. It measures compatibility via I/O type coercion, category Jaccard, and keyword co-occurrence. It detects dependencies between steps via linguistic markers. The compose stage gets the least evaluation in the paper. The paper focuses on the decompose-retrieve bottleneck.

The phase transition hiding in your agent’s skill library selection accuracy collapses at approximately 80 to 90 skills. The collapse happens due to semantic confusability, not library size. The two papers connect at the routing layer.
Your agent’s memory is making it a yes-man memory introduces a different failure mode in agents: sycophancy. Retrieval works but the agent agrees with the user’s prior beliefs regardless of facts.

This article was published on Agentic Up (https://agenticup.dev): practical guides for developers and founders building with AI agents. Reach me at [email protected].