Scaling the Horizon, Not the Parameters: Reaching Trillion-Parameter Performance with a 35B Agent

A research team has introduced Agents-A1, a 35-billion-parameter Mixture-of-Experts agentic model that matches the performance of trillion-parameter systems by scaling what they call the agent horizon, according to a preprint posted to arXiv on June 29 [1]. The model was trained using a three-stage recipe. The first stage applies full-domain supervised fine-tuning to align a base model with broad agentic behaviors. The second stage trains domain-level teacher models to capture specialized expertise. The third stage employs a multi-teacher domain-routed on-policy distillation technique with salient vocabulary alignment, designed to improve knowledge transfer across six heterogeneous domains and unify them into a single deployable student model [1][2]. The training infrastructure connects external knowledge, actions, observations, and verifier outcomes, producing agentic trajectories that average 45,000 tokens in length [1][2]. The authors describe this as scaling the agent horizon from two perspectives: scaling long-horizon trajectories and scaling heterogeneous agent abilities [2]. On long-horizon agent benchmarks, Agents-A1 posts leading results against trillion-parameter models such as Kimi-K2.6 and DeepSeek-V4-pro. It scores 56.4 on SEAL-0, 80.6 on IFBench, 46.4 on HiPhO, 79.0 on FrontierScience-Olympiad, and 56.8 on MolBench-Bind. It remains competitive on SciCode at 44.3, HLE at 47.6, and BrowseComp at 75.5 [1][2]. DeepSeek, one of the comparison models, is a Chinese AI company founded in July 2023 that develops large language models. Its DeepSeek-R1 model drew industry attention in January 2025 for delivering performance comparable to OpenAI's GPT-4 and o1 at a reported training cost of US$6 million, roughly one-sixteenth of GPT-4's estimated cost. DeepSeek incorporates mixture-of-experts layers to reduce training expenses and trained its models using weaker AI chips subject to export restrictions [6]. The Agents-A1 paper appears on arXiv, which since 2021 has integrated with Hugging Face Spaces to allow researchers to attach interactive demos directly to paper pages. The integration lets users try models in a browser without writing code, using open-source tools such as Gradio and Streamlit [3][4][5]. The preprint's abstract page includes links to code, data, and media associated with the article, as well as a Demos tab for community-built interactive applications [1][3][4].