Formalizing Numerical Analysis: An Agent Pipeline and Quality Audit Beyond Kernel Acceptance

A coding agent has formalized an entire textbook on numerical methods for ordinary differential equations in the Lean 4 proof assistant, and researchers have published a new framework showing that standard compilation checks dramatically overstate the quality of such machine-generated mathematics [1][2]. The work, submitted to the arXiv preprint repository in 2026, applies an agent pipeline to a branch of numerical analysis largely absent from mathlib, the central mathematical library for Lean 4 [1][2]. The textbook formalized is Numerical Methods for Ordinary Differential Equations, and the project was designed to stress the agent's capacity to develop new theory from scratch rather than relying on pre-existing library components [2]. The arXiv repository, which hosts the paper, was founded in 1991 and now receives approximately 24,000 submissions per month, serving as a primary distribution channel for pre-peer-review research in mathematics, physics, and computer science [6]. The researchers introduce a three-dimensional evaluation framework that moves beyond kernel acceptance — the Lean 4 compiler's verification that a proof is syntactically correct [2]. The three dimensions are semantic correctness, Mathlib reuse, and cross-file reuse, with the latter assessed using LLM-as-judge methods [1][2]. Large language models are machine learning systems trained on vast text corpora for natural language processing tasks [8]. When the framework was applied to the team's own formalization and to the released outputs of the RepoProver and M2F systems, it uncovered recurring unfaithful formalization patterns [2]. These included incomplete multi-part statements, added weakening hypotheses, and parameter restrictions — flaws that kernel acceptance entirely obscures [1][2]. The paper appears on an arXiv abstract page that features arXivLabs, a framework launched in 2020 to enable community collaborators to develop experimental tools that integrate directly with the repository [4]. arXivLabs projects, which include citation explorers and recommender systems, operate under guidelines requiring partners to share arXiv's values of openness, community, excellence, and user data privacy [4][5]. The arXivLabs program is currently on a temporary hiatus for new proposals while the development team focuses on modernizing arXiv's infrastructure and migrating systems to the cloud [3]. The researchers conclude that compilation-based metrics substantially overstate formalization quality and provide a reproducible audit methodology intended to support more rigorous evaluation of future autoformalization systems [1][2].