Symmetry-Aware Transformer Training for Automated Planning

A team of researchers has proposed a novel contrastive learning objective designed to make transformer models symmetry-aware, addressing a core weakness that has limited their use in automated planning, according to a paper posted on arXiv [1]. The paper, submitted on 11 August 2025 and last revised on 26 June 2026, tackles a known shortcoming of state-of-the-art models like PlanGPT, a decoder-only transformer that struggles when extrapolating from simple to complex planning problems [1][2]. The root cause, the authors argue, is problem symmetries: planning tasks can be expressed using arbitrary variable names that serve only as identifiers, leading to a combinatorial explosion of equivalent representations that pure transformers cannot learn from efficiently [2]. To compensate for the architecture’s lack of inductive bias, the researchers combined a contrastive learning objective with architectural improvements, enabling transformers to be trained for either plan-generation or heuristic-prediction [2]. The work was authored by Markus Fritzsche and posted on arXiv, the open-access e-print repository that has hosted over two million articles since its founding in 1991 and currently receives roughly 24,000 submissions per month [1][6]. The paper’s abstract states that results across multiple planning domains demonstrate the symmetry-aware training “effectively and efficiently addresses the limitations of PlanGPT” [2]. The initial submission weighed 605 KB, while the revised version is 565 KB [1]. The research appears on arXiv’s abstract page alongside community-developed tools offered through arXivLabs, a framework launched in 2020 that allows third-party collaborators to build experimental features such as bibliographic explorers and code finders directly on the site [5][4]. arXivLabs operates under guidelines that require partners to uphold the repository’s values of openness, community, excellence, and user data privacy, with collaborators receiving only minimal, anonymized data [5]. The framework is currently on a temporary hiatus for new proposals while the development team focuses on migrating arXiv’s systems to the cloud [3].