Boosting Self-Consistency with Ranking

A new method called Ranking-Improved Self-Consistency (RISC) reformulates how large language models select answers from multiple reasoning attempts, treating the final choice as a ranking problem rather than a simple majority vote [1]. The technique, detailed in a paper submitted to arXiv on June 3, 2026, addresses a known weakness in standard self-consistency approaches. These methods sample several reasoning paths and pick the most common answer, but majority voting often fails to recover correct answers that are already present among the samples [2]. RISC instead uses a lightweight LambdaRank model to score candidate answers [1]. The model relies on five features that capture answer frequency, semantic centrality, and reasoning-trace consistency [3]. The researchers evaluated RISC on three datasets under a range of test-time budgets [4]. Across these benchmarks, RISC consistently achieved a better accuracy-efficiency trade-off than standard self-consistency and other strong baselines, with particularly large gains on question answering tasks [5]. The paper’s analysis shows that each of the five features contributes meaningful improvements on its own, while their combination leads to additional performance gains [3]. The authors also conducted feature ablation studies and used SHAP analysis, revealing non-additive interactions between the features that explain why a learned combination outperforms any single handcrafted signal [4]. The design aligns with recent research suggesting that lightweight question-level and external-knowledge signals can provide useful reliability information without requiring expensive additional calls to a large language model [3]. On the PopQA and HotpotQA datasets, the performance gap between RISC and baseline methods appeared early and widened as the test-time budget increased; on the MATH500 dataset, the gap was also present [5].