You Don't Need to Run Every Eval
- company Hugging Face
- location arXivLabs
- model GPQA-D
- model HLE
- model MMLU-Pro
- product ARC-AGI-1
- product BenchPress
- product Codeforces
A new study finds that a frontier model’s scores across 133 benchmarks can be largely determined by just two numbers, suggesting that many costly evaluations may be redundant [1]. The researchers compiled a public score matrix of 84 models and designed a method called BenchPress to recover held-out scores to within 4.6 points [2]. The study, posted to arXiv on June 22, 2026, assembled a matrix of 2,604 cells, of which only 23.3 percent were filled, tracking 84 frontier models across 133 benchmarks [2]. The authors found the matrix is approximately rank-2, meaning a model’s performance across all benchmarks is largely determined by two underlying factors. Two factors already explain over 90 percent of the variation among models on the benchmarks they share [2]. Building on this finding, the team designed BenchPress, a logit-space rank-2 matrix completion method that recovers held-out scores to within 4.6 points and includes a confidence layer that indicates when each prediction can be trusted [2]. Using BenchPress, the researchers identified a subset of five benchmarks—GPQA-D, HLE, Codeforces, MMLU-Pro, and ARC-AGI-1—that can recover the rest of a model’s public scorecard to within 3.93 points. For a tighter inference budget, a cheaper set—GPQA-D, MMLU-Pro, Aider Polyglot, MATH-500, and AIME 2026—can predict a model’s evals to within 4.55 points [2]. The work arrives as the machine-learning community increasingly seeks ways to make research more reproducible and accessible. Since October 2021, Hugging Face Spaces has been used to build and share over 12,000 open-source machine learning demos, enabling anyone with a browser to try models without writing code [9]. In 2022, Hugging Face collaborated with arXiv to integrate these demos directly onto paper abstract pages through a Demo tab, allowing readers to explore results interactively [10]. The BenchPress authors have released the score matrix, the BenchPress code, and an interactive tool that predicts any model’s score on any benchmark [2].
research-papermodel-releasebenchmarkinfrastructure
Background sources we checked (10)
- arxiv.org ↗ A modern model release reports scores on 40+ benchmarks and the same evaluations were run many more times before it: to track training progress, compare design choices, and select the checkpoint for the release. But do we need to run every eval? We compile a public score matrix o…
- en.wikipedia.org ↗ "You Don't Own Me" is a pop song written by Philadelphia songwriters John Madara and David White and recorded by Lesley Gore in 1963, when she was 17 years old. The song was Gore's second most successful recording and her last top-ten single. Gore herself considered it to be her …
- en.wikipedia.org ↗ Octopart is a web-based search engine and data platform for electronic and industrial components. It aggregates catalog, pricing, availability, and technical parts data with related content like CAD models, datasheets, and eval kits from distributors and manufacturers online, all…
- en.wikipedia.org ↗ Killing Eve is a British black dramedy spy thriller television series produced in the United Kingdom by Sid Gentle Films for BBC America and BBC Three. The series follows Eve Polastri (Sandra Oh), a British intelligence investigator tasked with capturing psychopathic assassin Vil…
- en.wikipedia.org ↗ YouTube is an American online video sharing platform owned by Google. YouTube was founded by Chad Hurley, Jawed Karim, and Steve Chen, three former employees of PayPal. It is the second-most-visited website. Most content is generated by individuals, including collaborations betwe…
- en.wikipedia.org ↗ This is an incomplete list of Korean dramas, broadcast on nationwide networks KBS (KBS1 and KBS2), MBC, SBS; and cable channels JTBC, tvN, OCN, Channel A, MBN, Mnet and TV Chosun. The list also contains notable miniseries and web series broadcast on Naver TV, TVING, Wavve, Coupan…
- en.wikipedia.org ↗ BBC Radio 1's Big Weekend (R1BW) (previously known as One Big Weekend, for 2012 as Radio 1's Hackney Weekend, and for 2018 as BBC Music's Biggest Weekend) is a British music festival run by BBC Radio 1. It is held once a year, in a different location within the United Kingdom eac…
- huggingface.co ↗ Hugging Face Machine Learning Demos on arXiv Back to Articles ... # Hugging Face Machine Learning Demos on arXiv Published November 17, 2022 Update on GitHub Upvote 1 - - - - - Abubakar Abid abidlabs Follow …
- info.arxiv.org ↗ ## Hugging Face Spaces ... Hugging Face code repositories, About Hugging Face ... Collaborators: Abubakar Abid, Omar Sanseviero, Ahsen Khaliq, and the Hugging Face team ... Hugging Face Spaces includes links to demos created by the community or the authors themselves. By going to…
- huggingface.co ↗ Demos on Hugging Face Spaces allow a wide audience to try out state-of-the-art machine learning research without writing any code. Hugging Face and ArXiv have collaborated to embed these demos directly along side papers on ArXiv! ... Thanks to this integration, users can now find…
Sources
- export.arxiv.org — You Don't Need to Run Every Eval ↗