Compositional Skill Routing for LLM Agents: Decompose, Retrieve, and Compose

A new framework called SkillWeaver tackles the problem of getting large language model agents to compose multiple external tools for complex tasks, according to research posted to arXiv on June 16. The authors formalize the challenge as Compositional Skill Routing and release a benchmark of 300 queries spanning 2,209 real-world skills. LLM agents increasingly depend on external skills — reusable tool specifications — but real-world tasks often require chaining several skills together rather than calling a single one [1][2]. The researchers define Compositional Skill Routing as the process of decomposing a user query into atomic sub-tasks, retrieving the correct skill for each, and assembling an executable plan [1]. Their proposed solution, SkillWeaver, combines an LLM-based task decomposer, a bi-encoder skill retriever backed by FAISS indexing, and a dependency-aware directed acyclic graph planner [2]. To measure progress, the team built CompSkillBench, a benchmark containing 300 compositional queries drawn from 2,209 real MCP server skills across 24 functional categories, all sourced from the public MCP ecosystem [1][2]. MCP, or Model Context Protocol, provides a standardized way for models to discover and invoke tools. The benchmark exposes a clear bottleneck: standard LLM decomposition achieves only 34.2% category recall at the step level and 51.0% decomposition accuracy [2]. To improve decomposition, the authors introduce Iterative Skill-Aware Decomposition, or SAD, a retrieval-augmented feedback loop that repeatedly aligns the decomposition with the skills actually available in the library [1][2]. A single SAD iteration lifts decomposition accuracy from 51.0% to 67.7%, a relative gain of 32.7% with a Wilcoxon p-value below 10⁻⁶ [2]. Analysis conditioned on decomposition accuracy shows that correct granularity is a prerequisite for effective retrieval: category recall at rank one rises from 34% to 41% when decomposition accuracy equals one [2]. SkillWeaver also addresses efficiency. The framework reduces context-window consumption by more than 99% compared to loading full skill descriptions into the prompt [1][2]. In transfer experiments, where target categories were absent from the retrieval pool, the system still posted a 35.6% relative gain in decomposition accuracy, indicating generalization beyond the skills seen during training [2]. The paper appears on arXiv, an open-access repository that hosts preprints across physics, mathematics, computer science, and related fields [6]. Founded in 1991, arXiv now receives roughly 24,000 submissions per month and has served as the primary distribution channel for much of the machine-learning community’s fast-moving research [6].