AGORA: An Archive-Grounded Benchmark for Agentic Workplace Document Reasoning

A new benchmark called Agora tests whether large language models can reason across sprawling workplace document collections, a task that requires finding sparse evidence and reconciling inconsistent terminology. The strongest model evaluated reached only 59.4% accuracy, according to a paper posted to arXiv on June 23, 2026 [1]. The benchmark pairs 362 questions with eight domain collections containing 9,664 authentic documents and 372 million tokens, a scale that exceeds any model's context window and forces deliberate exploration rather than exhaustive scanning [1][2]. Agora was built using an agentic pipeline that combines cross-document task synthesis, obfuscation to prevent data leakage, and difficulty filtering [1][2]. Researchers evaluated eight models on the benchmark and found performance varied notably across domains, with no model exceeding the 59.4% accuracy mark [1][2]. The paper defines archive-grounded reasoning as the ability to locate sparse evidence across large, messy collections of workplace files while reconciling inconsistent terminology, units, and time conventions [1][2]. The work appears on arXiv, the open-access repository of electronic preprints that has hosted scientific papers since August 1991 and now receives roughly 24,000 submissions per month as of November 2024 [6]. The paper's abstract page features arXivLabs integrations, a framework launched in 2020 that allows community collaborators to develop tools such as the Bibliographic Explorer and CORE Recommender, which appear as tabs on article record pages [4][5]. arXivLabs requires partners to adhere to values of openness, community, excellence, and user data privacy, and collaborators receive only minimal, anonymized user data necessary for their features to function [4]. Large language models, the type of system tested on Agora, are machine learning models with many parameters trained on vast amounts of text for natural language processing tasks such as language generation [8]. The Agora benchmark is designed to stress-test these models in a setting that combines archive-groundedness, agentic exploration, and cross-domain coverage — dimensions the authors say existing benchmarks address only in isolation [1][2].