A Benchmark of State-Space Models vs. Transformers and BiLSTM-based Models for Historical Newspaper OCR

A new benchmark pits State-Space Models against Transformer and BiLSTM architectures for transcribing historical newspapers, finding that Mamba-based models can halve inference time while maintaining competitive accuracy on degraded text from the Bibliotheque nationale du Luxembourg. The study, submitted to arXiv on 1 April 2026 and revised on 23 June, addresses a persistent bottleneck in cultural-heritage digitization: end-to-end optical character recognition for long, degraded newspaper columns. Transformer-based recognizers, though dominant, carry quadratic complexity that limits efficient paragraph-level transcription and large-scale deployment [1]. The authors propose what they describe as the first OCR architecture built on State-Space Models, pairing a CNN visual encoder with bi-directional and autoregressive Mamba sequence modeling [2]. Experiments were run on newly released gold-standard annotations verified to more than 99% accuracy, alongside cross-dataset tests on Fraktur and Antiqua lines [2]. All neural models achieved character error rates around 2%, shifting the practical contest from raw accuracy to computational cost [1]. On severely degraded paragraphs, Mamba-based models recorded a 6.07% CER, compared with 5.24% for the DAN Transformer baseline, while running 2.05 times faster [2]. Memory scaling also favored the SSM approach: at 1,000 characters, Mamba memory usage grew by a factor of 1.26, versus 2.30 for the Transformer baseline [2]. The benchmark evaluated multiple decoding strategies—CTC, autoregressive, and non-autoregressive—under identical training conditions, and included off-the-shelf engines such as PERO-OCR, Tesseract OCR, TrOCR, and Gemini [1]. The authors have released code, trained models, and standardized evaluation protocols on GitHub to support reproducible research [2]. The paper is indexed on Hugging Face, where users can link models, datasets, and interactive demos to its arXiv page [4][5]. While the work focuses on historical newspapers, the efficiency gains it documents align with a broader push toward linear-time sequence models in machine learning. Large language models such as DeepSeek and Qwen have drawn attention for reducing training costs through architectural choices like mixture-of-experts layers, though those systems operate in different domains [7][9]. The SSM-OCR benchmark provides a controlled comparison specific to document transcription, offering practitioners a data point for choosing architectures when scaling up cultural-heritage projects.