Retrieved In-Context Principles from Previous Mistakes

A new teacher-student framework called Retrieved In-Context Principles (RICP) aims to improve the reasoning of large language models by having a teacher model analyze a student model's mistakes and generate customized corrective principles, according to a paper posted to arXiv [1]. The framework, detailed in a revised paper submitted in May 2026, addresses limitations in current methods that use principles derived from errors, which the authors say suffer from a lack of customization and inadequate error coverage [2]. In RICP, a teacher model analyzes the student model's mistakes to generate reasons and insights for preventing similar errors. These mistakes are then clustered by their underlying causes to develop task-level principles, which are intended to enhance error coverage [2]. During inference, the most relevant past mistakes for a specific question are retrieved to create question-level principles, improving the customization of the guidance provided [2]. The method is orthogonal to existing prompting strategies and does not require the teacher model to intervene during inference [1]. The researchers tested the framework across seven reasoning benchmarks and found it effectively enhanced performance when applied to various prompting strategies [1]. The paper was submitted by Hao Sun [1]. The revised version of the paper, v2, is 817 KB, down from the 1,590 KB v1 submission [1]. The work builds on the broader field of in-context learning, which adapts large language models to downstream tasks using correct input-output examples [2]. Large language models, such as those powering OpenAI's ChatGPT, have accelerated an AI boom marked by rapid investment and public attention since late 2022 [3]. These models can generate plausible-sounding but incorrect answers, known as hallucinations, a limitation that frameworks like RICP seek to mitigate by learning from prior mistakes [3]. Other alignment techniques, such as reinforcement learning from human feedback (RLHF), train a reward model from human preference data to improve an agent's policy, but sourcing high-quality preference data remains an expensive process [5]. The RICP framework offers an alternative approach by automating the error analysis and principle generation process without ongoing human annotation or teacher model involvement during inference [1][2].