Cost-Optimal LLM Routing with Limited User Feedback under User Satisfaction Guarantees

A new online routing algorithm for large language model applications, SLARouter, learns cost-optimal policies from sparse user feedback while providing theoretical guarantees for cost and service-level agreement compliance, according to a paper submitted to arXiv on 12 June 2026 [1][2]. The preprint, authored by Herbert Woisetschläger, was posted to the open-access repository arXiv at 08:50:46 UTC and runs to 160 KB [1]. arXiv, which began on 14 August 1991, hosts non-peer-reviewed e-prints across mathematics, physics, computer science, and related fields, and as of November 2024 was receiving about 24,000 submissions per month [6]. The paper addresses the tension between rising inference costs for LLM applications and user expectations codified in Service Level Agreements [2]. Existing cost-aware routing methods, the authors note, typically require complete feedback signals, offline training, and extensive per-workload tuning, and most lack SLA guarantees or adaptivity at inference time [2]. SLARouter instead learns a cost-optimal policy from the sparse, one-sided user feedback available in production systems [2]. The algorithm provides theoretical guarantees for both cost optimality and strict SLA compliance [2]. Across a range of LLM benchmarks, SLARouter satisfied SLA constraints without per-benchmark tuning and reduced operating cost by up to 2.2x over existing baselines [1][2]. The paper appears on arXiv under the machine learning category and is accompanied by experimental features from arXivLabs, a framework launched in 2020 that allows community collaborators to build tools on the platform while adhering to values of openness, community, excellence, and user data privacy [5]. Current Labs integrations include the Bibliographic Explorer for citation-tree navigation and the CORE Recommender for discovering open-access papers across repositories [4][5].