Scaling Laws for Agent Harnesses via Effective Feedback Compute

A new study proposes measuring language-model agent performance not by raw compute spending but by Effective Feedback Compute (EFC), a metric that counts only feedback that is informative, valid, non-redundant, and retained for later decisions [1]. Agent harnesses govern how language models call tools, verify intermediate states, store memory, and revise solutions, making them central to system performance [1]. Current test-time scaling analyses typically parameterize this process by raw expenditure — tokens, tool calls, operations, wall time, or cost — without distinguishing useful feedback from redundant or unstable interaction [1]. The researchers introduce EFC as a trace-level scaling coordinate that credits feedback only when it meets four criteria: informativeness, validity, non-redundancy, and retention for subsequent decisions [1]. They also normalize EFC by task demand when comparing tasks with different feedback requirements [1]. In controlled scaling experiments, raw tokens and tool calls explained limited variation in failure rates, with R² values of 0.33 and 0.42 respectively [1]. A multivariate SAS baseline reached an R² of 0.88, while Oracle-EFC and Estimated-EFC reached 0.94 [1]. Oracle-EFC normalized by task demand achieved an R² of 0.99 [1]. Matched-budget interventions demonstrated that improving feedback quality raised success rates from 0.27 to 0.90 while raw cost and tool calls remained fixed [1]. On mixed real traces, NRS-EFC normalized by task demand reached an R² of 0.92, while raw compute showed near-zero or negative fit [1]. The metric remained the best predictor in a prospective holdout, with an R² of 0.85 [1]. The findings arrive as prompt engineering and context engineering have become established practices for structuring inputs to generative AI models [3]. Techniques such as chain-of-thought prompting and retrieval-augmented generation aim to improve output accuracy, but the EFC framework suggests that the quality of feedback flowing back to the model during multi-step tasks may matter more than the volume of tokens or tool calls expended [3]. Deep learning architectures, including transformers, have been applied across computer vision, natural language processing, and other domains, where they have produced results comparable to and in some cases surpassing human expert performance [5]. The EFC study extends this line of inquiry by focusing on the efficiency of the harness layer that orchestrates model interactions rather than on model architecture itself [1]. The results indicate that harness scaling is governed less by how much computation is spent than by how efficiently raw budget is converted into durable, task-sufficient feedback [1]. The researchers validated EFC-based coordinates across synthetic controllable tasks, executable code tasks, real benchmark traces, held-out splits, and a prospective validation batch [1].