Token-Level LLM Collaboration via FusionRoute

A new framework called FusionRoute proposes a token-level method for coordinating multiple large language models, outperforming existing collaboration techniques across several benchmarks, according to a paper posted on arXiv [1]. The approach, detailed by researcher Nuoya Xiong, addresses a persistent tension in the field: general-purpose models require massive scale to perform well across domains, while smaller, specialized models are efficient but fail to generalize [1][2]. FusionRoute deploys a lightweight router that selects the most suitable expert model at each decoding step and simultaneously contributes a complementary logit to refine the chosen expert’s next-token prediction [1][2]. The paper provides a theoretical analysis arguing that pure expert-only routing is fundamentally limited unless strong global coverage assumptions hold, and that the addition of a trainable complementary generator expands the effective policy class to recover optimal value functions under milder conditions [2]. The framework was tested across both Llama-3 and Gemma-2 model families [1][2]. Gemma is a series of source-available large language models developed by Google DeepMind, with the first version released in February 2024 and Gemma 2 following in June 2024 [7]. DeepMind, a subsidiary of Alphabet Inc., has previously created models such as AlphaGo and AlphaFold and now leads development of Google’s Gemini family of large language models [4]. On benchmarks spanning mathematical reasoning, code generation, and instruction following, FusionRoute outperformed sequence-level and token-level collaboration methods, model merging, and direct fine-tuning [1][2]. The system also remained competitive with domain experts on their respective tasks [1][2]. The paper was first submitted on January 8, 2026, and revised most recently on June 12, 2026 [1].