Cliff Tokens: Identifying Single-Token Failure Triggers in LLM Mathematical Reasoning

A team of researchers has identified single tokens inside large language models that act as failure triggers during mathematical reasoning, offering a new lens on why models sometimes produce wrong answers even when they are capable of solving a problem correctly. The study, posted to arXiv on June 24, introduces the concept of a "cliff token" — a specific token where the model's probability of reaching the correct answer drops sharply [1][2]. The drop is measured by an adaptive threshold that scales with the local token-wise potential, using a one-sided two-proportion z-test [2]. Prior work analyzed failure at the step, chunk, or sentence level, or examined tokens only after failure had already occurred, but did not pinpoint the exact token that initiates the shift toward an incorrect result [1][2]. The researchers tested their method across seven models and three mathematical reasoning benchmarks: GSM1K, MATH500, and AIME 2025 [1][2]. When the first cliff token in a reasoning trace was deleted and the model resampled from that point, the pass@64 recovery rate reached 1.0. When the cliff token was kept, recovery remained between 0.71 and 1.00 [1][2]. The paper also proposes a taxonomy that divides cliff tokens into three categories: deterministic, uncertain, and sampled-off cliffs, defined by whether the token was the greedy choice and by its entropy [1][2]. Each type exhibits distinct probabilistic characteristics, and the taxonomy was found to generalize across model scales [2]. To test whether the taxonomy has practical value, the authors applied single-token preference optimization at cliff positions, a method they call Cliff-DPO. Training on the GSM8K dataset and evaluating on other benchmarks, Cliff-DPO improved accuracy by up to +6.6 [1][2]. Optimization at uncertain and sampled-off cliffs yielded reasoning improvements, while optimization at deterministic cliffs did not [1][2]. The work arrives amid broader efforts to understand and improve the reliability of large language models. LLMs are language models with many parameters, trained with self-supervised learning on vast amounts of text [8]. Companies such as DeepSeek, a Chinese AI firm founded in 2023, and Alibaba Cloud, which develops the Qwen family of models, have released open-weight models that compete with proprietary systems from OpenAI and Meta [7][9]. DeepSeek's R1 model, launched in January 2025, drew attention for performance comparable to GPT-4 and o1 at a reported training cost of US$6 million, far below the US$100 million cost cited for GPT-4 in 2023 [7]. The cliff-token study was shared on arXiv, a preprint server that accounts for roughly 95 percent of paper URLs linked by Hugging Face users in their repositories [4]. Hugging Face and arXiv have collaborated to embed interactive demos directly alongside papers on arXiv abstract pages, allowing readers to test models without writing code [5].