GeoWorld-VLM: Geometry from World Models for Vision-Language Models

A new framework called GeoWorld-VLM aims to fix a persistent weakness in vision-language models: their inability to reliably judge elementary spatial relations such as “left of,” “on,” or “behind.” The method distills geometric structure from frozen video world models directly into the visual pathway of existing VLMs, without retraining the language backbone. Modern Vision-Language Models achieve strong semantic recognition but remain brittle on basic spatial relations, a failure the authors trace to the visual pathway compressing or discarding critical 3D structural cues before language reasoning begins [1]. GeoWorld-VLM addresses this by fine-tuning only the image encoder and multimodal projector, aligning post-projector image features with intermediate representations from a frozen camera-conditioned video world model [1]. The world-model teacher converts static images, a prompt, and a sampled camera trajectory into a synthetic multi-view spatial signal, and training combines spatial answer supervision, teacher-student feature alignment, and a preservation anchor to the original VLM [1]. Because the language model stays frozen, the framework preserves the original model’s linguistic capabilities while attributing spatial improvements to the enhanced visual pathway [1]. The researchers applied GeoWorld-VLM to two distinct VLM architectures and observed consistent gains across both backbones [1]. On the What’sUp and VSR benchmarks, the method improved performance by approximately 4 percent, suggesting that world-model-guided visual alignment generalizes across model structures and spatial reasoning datasets [1]. The work arrives amid broader efforts to give VLMs spatial imagination. A related framework, World2VLM, treats a generative world model as a training-time teacher, synthesizing geometrically aligned future views from an anchor observation and a parameterized camera trajectory to provide structured supervision for both forward and inverse spatial reasoning [5]. World2VLM delivers consistent improvements over its base model on benchmarks including SAT-Real, SAT-Synthesized, and VSI-Bench, and it outperforms test-time world-model-coupled methods while eliminating the need for expensive inference-time generation [5]. GeoWorld-VLM’s authors note that ablations show prompt- and camera-aware intermediate world-model features are most effective, and the method requires no world-model inference at test time [3]. The paper was submitted to arXiv on 15 May 2026 and revised on 11 June 2026 [1]. The broader research context includes work such as GeoWorld, a geometric world model that maps latent representations onto hyperbolic manifolds to preserve hierarchical structure for multi-step planning [9], underscoring a growing interest in embedding geometric principles into learned visual representations.