Think-at-Hard: Selective Latent Iterations to Improve Reasoning Language Models

A research team has proposed Think-at-Hard, a looped transformer model that selectively skips latent iterations to improve reasoning in large language models, according to a paper posted on arXiv [1]. Large language models often struggle with reasoning tasks when operating under parameter constraints [1]. Looped transformers, which perform multiple latent iterations to refine each token beyond a single forward pass, have been used to address this limitation [1]. However, researchers led by Tianyu Fu identified a phenomenon they call latent overthinking: most token predictions are correct after the first pass but can be revised into errors during later iterations [1]. An oracle iteration policy that could perfectly decide when to skip iterations boosted performance by up to 7.3% [1]. Motivated by this finding, the team built Think-at-Hard, which employs a lightweight neural decider to trigger latent iteration only for tokens likely to be incorrect after the standard forward pass [1]. During latent iterations, depth-aware Low-Rank Adaptation modules shift the objective from general next-token prediction to focused hard-token refinement [1]. A duo-causal attention mechanism extends attention from the token sequence dimension to an additional iteration depth dimension, enabling cross-iteration information flow with full sequential parallelism [1]. Experiments across nine benchmarks covering math, question answering, and coding tasks showed consistent gains [1]. With identical parameter counts, Think-at-Hard outperformed always-iterate baselines by 3.8-4.4% while skipping iterations on 93% of tokens [1]. It exceeded single-iteration Qwen3 baselines by 3.0-3.8% [1]. When allowing fewer than 3% more parameters from the LoRA modules and decider, the gains increased to 5.3-6.2% and 6.1-6.8%, respectively [1]. The code has been made available on GitHub [1].