CELEUS: Certifiable and Efficient LLM Evaluation via E-Processes

A new framework called CELEUS aims to make the evaluation of large language models more statistically rigorous and efficient, according to a preprint posted on arXiv. The method promises to deliver trustworthy confidence intervals while requiring far fewer test samples than current approaches. The paper, submitted on 18 June 2026 and revised on 26 June 2026, addresses a core weakness in how LLM performance is measured [1]. Existing certifiable evaluation techniques sequentially select test samples and update confidence intervals until a desired precision is reached, typically claiming 95% coverage [1]. However, the authors argue these methods are not generally “anytime-valid,” meaning the stated coverage can break down when intervals are repeatedly updated and used to decide when to stop testing [1]. To close this gap, the researchers propose CELEUS, a framework that uses E-processes to construct confidence intervals that remain valid at any stopping time [1]. The system combines two techniques: uncertainty-guided sampling, which selects the most informative samples for evaluation, and surrogate-assisted approximations for samples that are not directly evaluated [1]. The authors prove that the resulting signals are unbiased for the evaluation score conditional on past data, which underpins the statistical guarantees [1]. In experiments, CELEUS reached the target precision using 54–62% fewer evaluated samples than baseline methods while preserving anytime-valid coverage [1]. The paper also includes a theoretical analysis showing that the confidence intervals can shrink at a near-parametric rate, up to logarithmic factors, and derives an oracle variance-optimal sampling rule that motivates the practical uncertainty-guided approach [1]. The work appears on arXiv, an open-access repository that hosts preprints across physics, mathematics, computer science, and related fields [6]. Founded in 1991, arXiv now receives roughly 24,000 submissions per month and has surpassed two million total articles [6]. Papers on the platform are moderated but not peer-reviewed, a distinction that places the burden of verification on the research community [6]. The repository also supports community-built tools through its arXivLabs framework, which allows third-party developers to create features such as citation explorers and code finders that appear on article pages [5]. These integrations are governed by guidelines that require partners to uphold arXiv’s values of openness, community, excellence, and user data privacy [5].