Rethinking the Pointer Loss in Table Structure Recognition: Geometry-Aware Pointer Loss for Spatial Locality

A new loss function for table structure recognition models targets a specific failure pattern: nearly four-fifths of pointer network errors occur between cells that sit close together on a page, according to research published on arXiv [1]. The work, posted June 17, 2026, introduces Geometry-Aware Pointer (GAP) Loss, a modification to the standard cross-entropy objective used in pointer networks for Table Structure Recognition (TSR) [1]. Pointer networks predict HTML tag sequences aligned to detected text regions, an approach that has yielded strong results on benchmark datasets [2]. However, the authors' error analysis showed that 79.6% of mistakes involve spatially adjacent cells — those within a Manhattan distance of 2 [2]. Standard cross-entropy loss treats all incorrect candidates equally, regardless of their physical position on the table grid [1]. GAP Loss reweights the objective using inverse distance weighting, so that immediate neighbors of the correct cell receive stronger gradient signals than distant cells [2]. The modification requires no changes to model architecture and adds zero computational cost at inference time [1]. Experiments on the PubTabNet and SynthTabNet datasets demonstrated that GAP consistently reduces adjacent-cell errors and achieves new state-of-the-art performance [2]. The code has been released on GitHub under the repository teamreboott/GAP [2]. The paper appears on arXiv, the open-access e-print repository that hosts preprints across physics, computer science, mathematics, and related fields [6]. As of November 2024, the platform receives roughly 24,000 submissions per month and has surpassed two million total articles [6]. The research falls within the Computer Vision and Pattern Recognition category and is accessible through arXiv's standard abstract page, which also surfaces experimental community tools via the arXivLabs framework [4][5]. Those tools include bibliographic explorers, code finders, and recommender systems developed by third-party collaborators under arXiv's values of openness, community, and user data privacy [4].