ACC: Compiling Agent Trajectories for Long-Context Training

A new training method called Agent Context Compilation (ACC) repurposes the sprawling interaction logs of AI agents to teach large language models long-context reasoning without additional human annotation, according to a paper posted to arXiv in May 2026 [1]. The approach addresses a known limitation in how AI agents are typically trained. When an agent solves a problem — invoking tools and receiving observations across many turns — the evidence needed to answer the original question is scattered throughout these turns, requiring integration of distant context segments [2]. Standard supervised fine-tuning masks those tool responses and only trains the model on turn-level tool selection, creating what the researchers call a supervision blind spot where these scattered signals go unused [2]. ACC converts trajectories from search, software engineering, and database querying agents into long-context question-answer pairs that combine the original question with tool responses and environment observations gathered across multiple turns, training the model to answer directly without tool use [2]. The researchers validated ACC on two benchmarks designed to test long-range dependency modeling. MRCR requires cross-turn coreference resolution, while GraphWalks demands graph traversal over extended contexts [2]. Training the Qwen3-30B-A3B model with ACC yielded an MRCR accuracy of 68.3, an improvement of 18.1 points, and a GraphWalks accuracy of 77.5, an improvement of 7.6 points [2]. Those results are comparable to Qwen3-235B-A22B, a model roughly eight times larger [2]. The paper, authored by Qisheng Su and colleagues, was initially submitted on 21 May 2026 as a 652 KB manuscript and revised on 14 June 2026 at 647 KB [1]. Crucially, the gains on long-context tasks did not come at the expense of general capabilities. The ACC-trained model preserved its performance on standard benchmarks including GPQA, MMLU-Pro, AIME, and IFEval [2]. A mechanism analysis further revealed that the model exhibited task-adaptive attention restructuring and expert specialization after ACC training [2]. The method is designed to be combined with any existing long-context extension or training approach, providing a scalable source of supervised fine-tuning data without requiring costly long-document curation or heuristic context synthesis [2].