Elastic Queries Reinforcement Learning: Self-Aware Policy Execution for VLA Models

A new framework called Elastic Queries Reinforcement Learning (EQRL) aims to make vision-language-action (VLA) models for robot manipulation more efficient by varying the computational effort per query based on task difficulty, according to a paper submitted in 2026 [1][2]. VLA models, which generate robot actions from visual and language inputs, are typically run with fixed inference and replanning schedules [1][2]. This approach ignores the fact that some manipulation states, such as those involving contact or uncertainty, require more computation and faster feedback, while simpler states can be handled with fewer inference steps and longer open-loop execution [2]. The EQRL framework addresses this by making each policy query elastic, jointly selecting the latent input, denoising budget, and action chunk length through a lightweight latent-schedule adaptor [1][2]. The underlying VLA model is not fine-tuned [2]. To determine state difficulty, EQRL trains a critic over the joint latent-schedule action and derives a difficulty signal from the disagreement within a critic ensemble [1][2]. This signal directs computational resources toward harder states, while a learned residual enables task-driven corrections [2]. The framework formulates variable chunk execution as query-level macro-action reinforcement learning with chunk-dependent discounting and an amortized number-of-function-evaluations budget [2]. In tests across both simulation and real-robot manipulation tasks, EQRL reduced amortized inference cost while maintaining or improving task success rates [1][2]. The paper was posted on arXiv, an open-access repository for electronic preprints that, as of November 2024, receives about 24,000 submissions per month and hosts over two million articles [6]. The work appears in the Computer Science and Robotics section of the repository [1].