SpatialWorld: Benchmarking Interactive Spatial Reasoning of Multimodal Agents in Real-World Tasks

A new benchmark called SpatialWorld reveals that even the most advanced multimodal AI agents struggle with interactive spatial reasoning, with the top model succeeding in fewer than one in five real-world tasks [1]. The benchmark, detailed in a paper submitted to arXiv on June 8, integrates eight different simulation backends under a single protocol to test how well agents perceive and act in physical environments [1]. It comprises 760 human-annotated tasks spanning household routines, travel, and social collaboration [1]. Unlike older evaluations that rely on static question-answering, SpatialWorld requires agents to operate under vision-only partial observability, gathering egocentric visual evidence and issuing decisions through a text-based action interface [1]. Each task includes a human-validated initial state, a reference trajectory, and a terminal-state verifier to ensure reliable scoring [1]. Researchers tested 15 advanced agents and found that robust spatial task solving remains elusive [1]. GPT-5, the strongest model evaluated, achieved an average task success rate of just 17.4% [1]. The leading open-source model, Qwen-3.5 — a member of the Qwen family developed by Alibaba Cloud and distributed under open-source licenses [9] — reached 14.1% [1]. The paper also documents a clear mismatch between task success and execution efficiency, along with significant performance variations across different domains [1]. The results underscore a gap between the capabilities measured by conventional language model benchmarks and the demands of interactive, real-world settings. Standard benchmarks typically provide static datasets and measure accuracy on tasks such as text classification or question answering [4]. By contrast, SpatialWorld requires agents to build and update an internal representation of a dynamic environment — a concept known in artificial intelligence as a world model, which simulates physics, object interactions, and causality to enable planning without constant real-world trial and error [5]. Foundation models, including large language models like the GPT series, are trained on vast datasets and can be adapted for many use cases, but their performance on interactive spatial tasks suggests that scaling data and parameters alone does not confer robust physical reasoning [6]. The benchmark arrives as the field grapples with rapidly rising scores on harder evaluation suites. The Stanford HAI 2025 AI Index Report noted that on benchmarks introduced in 2023, model performance jumped by 18.8, 48.9, and 67.3 percentage points on MMMU, GPQA, and SWE-bench respectively within a single year [3]. SpatialWorld’s low success rates indicate that interactive spatial understanding remains a frontier where current models have substantial room for improvement [1].