An Effective Router for Vision-Language Model Selection

Researchers have introduced ARMS, a router designed to select the most suitable vision-language model for a given query, addressing a gap in specialized data and model adaptability [1]. The system, detailed in a paper submitted to arXiv on June 8, 2026, tackles the challenge users face when choosing among numerous vision-language models (VLMs) with differing performance and resource demands [1]. The authors identify three primary obstacles: a lack of specialized data, ineffective feature representation, and a rigid model space that makes adaptation costly [1]. To overcome the first, they constructed a multimodal dataset containing the outputs of seven mainstream VLMs on 32,626 unique image-text queries [1]. ARMS enhances input signals by incorporating VLM profiles and uses a simple architecture to improve how queries and VLM capabilities are represented [1]. The router is notably compact at only 800 million parameters, yet the paper reports it can outperform commercial models such as GPT-4o, which are hundreds of times larger in scale [1]. This performance paradox, where smaller, specialized models can match or exceed larger generalist systems, has been previously observed in language models and is a driving factor behind routing research [1]. Large language models, the broader technology family that includes GPT-4o, are typically based on transformer architectures and are pre-trained to predict the next word before being fine-tuned for specific tasks [3]. The rapid improvement of such models has been documented by the Stanford HAI 2025 AI Index Report, which noted that on harder benchmarks introduced in 2023, model scores rose sharply within a single year, with performance increasing by 18.8, 48.9, and 67.3 percentage points on the MMMU, GPQA, and SWE-bench testing suites respectively by 2024 [3]. To address the issue of adapting to new VLMs, the ARMS framework proposes two extension training strategies: incremental training and independent training [1]. Experimental results on both in-distribution and out-of-distribution test sets demonstrated the effectiveness of the approach [1]. The paper's code, models, and datasets have been made available in an anonymous repository [1].