Know Before You Fetch: Calibrated Retrieval-Budget Allocation for Retrieval-Augmented Generation

A new study proposes an adaptive retrieval-augmented generation method that lets language models decide when to fetch external context, rather than retrieving a fixed number of passages for every query. The approach calibrates the model's internal confidence signals into probabilities of correctness, enabling graded retrieval-budget allocation across three question-answering benchmarks. Standard retrieval-augmented generation systems retrieve a fixed number of passages for every query, a practice the researchers describe as wasteful when the model already knows the answer and potentially harmful when irrelevant passages distract the reader [1][2]. The proposed method, detailed in a paper submitted to arXiv on 29 June 2026, reframes adaptive RAG as a calibrated retrieval-budget allocation problem: for each query, the system decides whether to answer without retrieval, retrieve a compact context of one passage, retrieve a full context of five passages, or abstain entirely [1][2]. The core contribution is a probability interface that calibrates sequence log-probability and prefix-logit uncertainty signals into probabilities of correctness, rather than introducing a new raw uncertainty metric [1][2]. The researchers then use these calibrated probabilities to drive graded context selection, selective abstention, and explicit latency-versus-token trade-offs [2]. Diagnostic out-of-fold calibration produced substantial improvements in probability quality across three datasets. On TriviaQA, expected calibration error dropped from 0.275 to 0.062; on Natural Questions, it fell from 0.643 to 0.009; and on MS MARCO, it declined from 0.711 to 0.031 [1][2]. The graded retrieval approach improved full-context and passage-budget frontiers for both the authors' signal and a TARG-style baseline using prefix entropy and margin [1][2]. Retrieval-call area-under-curve metrics remained essentially tied with binary gating, because retrieving even a single passage still counts as a retrieval call [1][2]. Held-out threshold experiments across train, validation, and test splits produced deployable operating points [1][2]. A measured cost model at matched-accuracy frontier operating points revealed that gating is not universally faster: latency increased by roughly 27% on Qwen3-8B but decreased by roughly 8% on Qwen3-32B [1][2]. The paper frames calibrated confidence as a reusable interface for allocating retrieval budget under task and system constraints, rather than a one-size-fits-all acceleration technique [1][2].