MA-ProofBench: A Two-Tiered Evaluation of LLMs for Theorem Proving in Mathematical Analysis

A new benchmark, MA-ProofBench, has been introduced to evaluate how well large language models perform automated theorem proving in mathematical analysis, a domain largely absent from existing formal reasoning tests. The benchmark reveals that even the strongest model, GPT-5.5, solves only a small fraction of its problems. The benchmark, detailed in a paper submitted to arXiv on June 11, 2026, contains 200 formalized theorems spanning 6 core topics and 27 subcategories, including measure and integration theory, complex analysis, and functional analysis [1][2]. The problems are split into two tiers: 100 at an undergraduate level and 100 at a Ph.D. qualifying level [1]. Each problem was constructed through a human-led, LLM-assisted formalization pipeline and then independently reviewed by experts to ensure mathematical fidelity [2]. Language model benchmarks are standardized tests used to compare model capabilities across tasks such as reasoning and generation [3]. MA-ProofBench is positioned as the first such benchmark dedicated to formal theorem proving in mathematical analysis, an area the authors say existing tests neglect in favor of easier-to-formalize fields like algebra and elementary number theory [2]. Researchers evaluated a range of recent general-purpose reasoning models and formal theorem provers on the new benchmark [1]. Performance was uniformly low. The best-performing model, GPT-5.5, achieved a Pass@8 score of 16% on the undergraduate-level problems and just 5% on the Ph.D.-level set [1][2]. Most other models scored near 0% on the more difficult tier [2]. The paper identifies two dominant failure modes: Mathlib hallucinations, where models fabricate or misuse components of the Lean mathematical library, and incomplete proofs that fail to reach a valid conclusion [1][2]. A separate evaluation on a natural-language version of the same problems exposed a clear gap between informal and formal reasoning capabilities [2]. The release of MA-ProofBench adds a new dimension to the landscape of AI evaluation, which already includes benchmarks for tasks such as quantum circuit generation [4]. The paper is indexed on arXiv and can be linked to models, datasets, or demo Spaces on platforms like Hugging Face, where community members can discuss results and share related artifacts [5][6][7]. The benchmark is intended to serve as a reference for tracking progress in formal mathematical reasoning within advanced domains [2].