Task-Instructed Causal Routing of Vision Foundation Models for Multi-Task Learning

Researchers have proposed TIGER, a framework that coordinates multiple vision foundation models for multi-task dense prediction, addressing the limitation of single models in capturing diverse visual representations.

Vision foundation models (VFMs) have shown robustness and transferability across various visual tasks, but each model has strong inductive biases shaped by its pre-training objective and data domain^[1]. A single VFM often struggles to capture the diverse visual representations required across multiple dense prediction tasks. To address this, TIGER leverages natural-language task instructions to guide a routing network that assigns token-level expert weights conditioned on task semantics. It also introduces a counterfactual loss that aligns routing decisions with each expert's causal contribution. TIGER was evaluated on two multi-task dense prediction benchmarks, NYUD-v2 and Pascal Context, where it outperformed recent multi-task learning baselines while keeping all VFMs frozen^[1]. Meanwhile, a separate evaluation of five state-of-the-art Visual Navigation Models (VNMs) revealed systematic limitations in their ability to navigate, including frequent collisions and goal prediction errors. The evaluation was conducted across two robot platforms and five environments, using path-based metrics and vision-based goal-recognition scores, as well as controlled image perturbations like motion blur and sunflare^[2].