RT-VLA: Real-Time Vision-Language-Action Models via Knowledge Distillation

A new lightweight vision-language-action model called RT-VLA achieves real-time autonomous driving performance by distilling a larger teacher model, cutting inference time by up to 44.8 times while preserving reasoning and explainability, according to research posted on arXiv [1]. Vision-Language-Action (VLA) models jointly handle visual perception, language reasoning, and action prediction for end-to-end autonomous driving, but their large backbones create inference latency that blocks real-world deployment [1]. The RT-VLA model addresses this by transferring the capabilities of the state-of-the-art SimLingo teacher into a compact student through multi-level supervised distillation [1]. The distilled model preserves language-based reasoning and supports post-hoc explanation through offline language analysis of safety-critical driving moments, without adding latency to real-time control [1]. Compared to the SimLingo teacher, RT-VLA maintains competitive closed-loop driving and language reasoning performance while reducing inference time by 44.8X in vision-only mode and 7.9X in vision-plus-language mode [1]. Knowledge distillation, the technique underpinning RT-VLA, has parallels in other scientific domains. In computational catalysis, researchers have explored transfer learning to improve model performance when consolidating data from multiple computational methods, including training on a larger dataset and fine-tuning on a smaller one [4]. The small-molecule and drug-discovery communities have similarly used transfer learning to move between varying levels of electronic-structure calculations or between related tasks [4]. The RT-VLA results suggest that supervised distillation is a practical approach for building real-time, explainable VLA-style autonomous driving models [1]. The work appears on arXiv under the Computer Vision and Pattern Recognition category, submitted on 12 June 2026 [1].