Beyond Monolingual Deep Research: Evaluating Agents and Retrievers with Cross-Lingual BrowseComp-Plus

A new benchmark called XBCP tests whether deep research agents can find and use evidence written in a language different from the user's query, according to a paper submitted to arXiv on 13 Jun 2026 [1]. The benchmark preserves English questions and answers but varies the languages of supporting documents across 12 languages [2]. Deep research agents are systems designed to search for evidence, reason over retrieved sources, and produce grounded answers [1]. Existing browsing benchmarks largely assume that the user's query and the supporting evidence are written in the same language, leaving open whether agentic search systems can operate when relevant evidence appears in another language [2]. XBCP, short for Cross-lingual BrowseComp-Plus, is a controlled benchmark that addresses this gap by instantiating two complementary settings [2]. In the cross-lingual setting, each query is paired with evidence in a single assigned language. In the multilingual setting, the full evidence corpus is distributed equally and randomly across 12 languages spanning high-resource and low-resource regimes [2]. The paper evaluates four deep research agents using sparse and dense multilingual retrievers, measuring answer accuracy, evidence recall, search behavior, calibration, citation fidelity, and oracle retrieval [1]. Results reveal substantial degradation when evidence is translated. Even strong, dense retrievers lose evidence recall, and agents become less calibrated and cite evidence less reliably [2]. Accuracy remains lower even when all gold evidence is supplied directly, suggesting that cross-lingual deep research exposes both retrieval failures and an independent, agent-side difficulty in integrating language-mismatched evidence [2]. The paper was posted on arXiv, an open-access repository of electronic preprints that, as of November 2024, receives about 24,000 submissions per month [6]. arXiv hosts papers in fields including computer science, mathematics, and physics, and is not peer reviewed [6]. The XBCP paper appears under the Computation and Language category [1]. arXiv also supports experimental community projects through arXivLabs, a framework that allows collaborators to develop and share new features directly on the site [4]. arXivLabs projects, which appear as tabs on article record pages, include tools such as the Bibliographic Explorer for navigating citation trees and the CORE Recommender for discovering related open-access papers [5]. arXiv has stated that third-party collaborators receive only minimal and anonymized user data, and any other use is prohibited without written consent [4].