EffGen: Enabling Small Language Models as Capable Autonomous Agents

A new open-source agentic framework called EffGen is designed to run capable autonomous agents on small language models (SLMs) locally, addressing the high token costs and privacy concerns tied to large-model APIs, according to a paper posted on arXiv [1]. The framework, released under the Apache 2.0 License, makes four contributions: enhanced tool-calling that compresses input prompts by up to 70-80 percent, averaging 57 percent across benchmarks; intelligent task decomposition; complexity-based routing using five factors; and a unified memory system combining short-term, long-term, and vector-based storage [1][2]. EffGen also unifies multiple agent protocols — MCP, A2A, and ACP — for cross-protocol communication [1][2]. In tests across 13 benchmarks, EffGen outperformed LangChain, AutoGen, and Smolagents, recording higher success rates, faster execution, and lower memory usage [1][2]. The paper’s authors, including Gaurav Srivastava, report that prompt optimization and complexity routing exhibit complementary scaling behavior: optimization benefits SLMs more, delivering an 11.2 percent gain at 1.5 billion parameters versus 2.4 percent at 32 billion parameters, while routing benefits larger models more, with a 3.6 percent gain at 1.5 billion parameters compared to 7.9 percent at 32 billion parameters [1][2]. Most existing agentic systems are built for large language models such as GPT, Claude, and Gemini, which are accessed via API calls [1][2]. That architecture imposes per-token fees and raises data-privacy risks for sensitive applications [1][2]. EffGen’s local-deployment design sidesteps those constraints by targeting SLMs that can run on-device [1][2]. The code is available on GitHub, the Python package can be installed via pip, and documentation is hosted at the project’s website [1][2]. The paper was submitted on 31 January 2026 and revised on 14 June 2026 [1]. While the primary source details the framework’s architecture and benchmark results, the broader research landscape includes work on transfer learning across datasets and the use of agent protocols in other domains, though those studies do not directly evaluate EffGen [3][4][5].