Looking Is Not Picking: An Attention-Segment Account of Tool-Selection Failures in LLM Agents

Large language model agents that select the wrong tool are not failing because they overlook the correct option in a crowded list, according to a new preprint. The models attend to the right tool 80% of the time but still choose incorrectly, pointing to a breakdown at the decision readout rather than at the input stage [1][2]. The paper, posted to the arXiv preprint repository on June 15, 2026, analyzes failures on the Berkeley Function Calling Leaderboard (BFCL) [1][2]. The authors find that when an LLM agent mis-calls a tool, the model’s attention is most strongly directed at the correct tool in 80% of cases, compared with a 21% chance baseline. The correct tool is the least-attended segment in only 10% of failures [2]. “It looks at the right tool and still picks wrong,” the abstract states, directly contradicting the common “crowded-harness” or “lost-in-the-middle” hypothesis [2]. The researchers isolate the bottleneck to the decision readout through three lines of evidence. First, repairing the prompt—by reordering or duplicating the correct tool definition—recovers at most 23% of failures. In contrast, interventions applied at the readout stage recover between 59% and 91% of failures [2]. Second, two distinct readout-side interventions—an additive attention-logit bias and a residual-stream steering vector—recover largely the same set of failures, with a per-task Jaccard similarity of 0.865 pooled and 0.79 to 0.91 per model. This representation-invariance suggests the bottleneck is localized to the readout regardless of which internal representation is modified [2]. Third, the authors propose a training-free, gold-free selector that uses per-segment attention to close most of the gap between a gold-free baseline and an oracle. On BFCL, the selector adds 11.9 points of function-name selection accuracy, compared with a 17.9-point oracle headroom. On the Seal-Tools benchmark, it adds 14.9 points. Every model tested showed a positive effect, with exact McNemar p-values at or below 8×10⁻⁴ [2]. The causal attention-bias dose-response is bidirectional and monotonic across 10 mask-honoring models ranging from 3 billion to 32 billion parameters. Across the full 0.5-billion to 32-billion span, the relationship remains correlational. The deployable selector was evaluated on five single-turn models and does not yet transfer to a multi-turn loop [2]. The work appears on arXiv, the open-access e-print repository that hosts preprints across physics, computer science, and related fields. Founded in 1991, arXiv surpassed two million articles by the end of 2021 and now receives roughly 24,000 submissions per month [6]. The paper is accompanied by experimental features under the arXivLabs framework, which allows community collaborators to build tools such as bibliographic explorers and code finders on the platform [4][5].