FinBalance: A Multi-Document Accounting Reconciliation Benchmark

A new benchmark called FinBalance tests whether large language models can perform multi-document accounting reconciliation, and the top-performing model achieved only 46% exact final-balance-sheet accuracy on a 710-record evaluation split, according to a paper posted to arXiv on June 14, 2026 [1][2]. FinBalance is built from source-document bundles spanning eight industries, three period types, and five difficulty levels [1][2]. The benchmark moves beyond existing financial-NLP evaluations, which the authors note mostly assess prepared artifacts such as filings, tables, or extracted values [2]. Real accounting begins earlier: source documents must be reconciled into cited journal entries, aggregated into a balance sheet, and checked for contradictions [2]. Human-authored business scenarios, accounting policies, tax and foreign-exchange treatments, document schemas, distractors, and inconsistency templates are composed by a deterministic generator whose ledger produces journal entries, balance sheets, and 23 inconsistency-code labels [2]. Six contemporary LLMs were evaluated on the 710-record split [1][2]. Four models exhibited a 26-to-41-percentage-point gap between BS_exact — the model’s reported balance sheet — and BS_recon, the balance sheet obtained by replaying its entries through the benchmark’s ledger [1][2]. The authors report that models often recover numerically plausible entries but fail to bind them to supporting documents and aggregate them consistently [2]. Citation-pressure prompting barely changed document-linking errors, while ledger-feedback ablations substantially improved reported balance sheets and exposed inconsistency-detection trade-offs [2]. Expert finance reviewers validated the benchmark design and labels [1][2]. The work arrives as research organizations continue to invest heavily in AI capabilities. Microsoft Research, for instance, has employed more than 1,000 computer scientists, physicists, engineers, and mathematicians and has spent an estimated $10 billion to $14 billion annually on research initiatives since 2010 [6]. Between 2010 and 2018, 154,000 AI patents were filed worldwide, with Microsoft holding the largest share at 20 percent [6]. The FinBalance benchmark was posted on arXiv, a preprint server that has long served as a primary distribution channel for computational research, including work from institutions such as Microsoft Research, which previously operated the now-shuttered Microsoft Academic search engine [7]. Vector databases, which store and retrieve embeddings of data in high-dimensional space, are increasingly used in retrieval-augmented generation and semantic search workflows [8]. The FinBalance authors’ finding that models struggle to bind entries to supporting documents underscores a challenge that vector-based retrieval systems are designed to address: locating semantically similar records across large document sets [2][8].