GeoDisaster: Benchmarking Orchestrated Agents for Operational Disaster Geo-Intelligence

A new benchmark called GeoDisaster aims to push remote-sensing vision-language models toward operational geo-intelligence by requiring tool-grounded spatial reasoning and structured, evidence-backed decisions, according to a paper posted to arXiv on June 15 [1][2]. The benchmark comprises 2,921 verified instances across 43 question types and five task families: deforestation monitoring, multi-hazard analysis, building-damage assessment, flood-safe routing, and Sentinel-1 SAR flood monitoring [1][2]. Each instance integrates heterogeneous Earth-observation and geographic information system evidence, including optical and synthetic aperture radar imagery, raster masks, vector geometries, road networks, and exposure layers [1][2]. The tasks span hazard detection, damage assessment, exposure estimation, and diagnostic report generation [1][2]. Ground-truth answers are derived from executable geospatial workflows and deterministic consistency checks, eliminating the need for language-model annotation [1][2]. The researchers also propose an orchestrated multi-agent framework equipped with 18 disaster-oriented tools [1][2]. Role-specialized agents coordinate through explicit execution contracts, aligned via a mechanism called Role-Contract Expectation Alignment, or RCEA [1][2]. RCEA combines failure-aware supervised fine-tuning with contract-grounded reinforcement learning over dense step-level signals [1][2]. Experiments indicate that GeoDisaster challenges existing RS-VLMs and agentic systems, while RCEA improves tool use, evidence grounding, state consistency, and decision generation [1][2]. The paper appeared on arXiv, the open-access e-print repository that hosts preprints across physics, mathematics, computer science, and related fields [6]. arXiv was founded in 1991 and, as of November 2024, receives approximately 24,000 submissions per month [6]. The repository is not peer-reviewed; papers are approved for posting after moderation [6]. The GeoDisaster paper is accompanied by links to code and data through services such as Hugging Face and Papers with Code, which are surfaced via arXivLabs integrations on the abstract page [1][4]. arXivLabs is a framework that allows community collaborators to develop and share experimental tools directly on the arXiv site, under guidelines that emphasize openness, community, excellence, and user data privacy [5].