PromptShift-CRC: Drift-Aware Conformal Risk Control for Foundation Models Under Prompt and Domain Shift

A new method called PromptShift CRC aims to keep foundation-model outputs within acceptable risk bounds even when the prompts users submit drift away from the data used to calibrate the system, according to a paper posted to arXiv on 14 Jun 2026 [1]. Foundation models are increasingly deployed in environments where the topics, users, and policies governing prompts can shift rapidly, making fixed calibration risky [1][2]. Standard conformal prediction and conformal risk control techniques provide model-agnostic error guarantees, but they rely on the assumption that future data will resemble the calibration data [2]. When that assumption breaks, the risk controls can fail. The paper, hosted on the arXiv preprint repository — which as of November 2024 was receiving roughly 24,000 submissions per month — introduces a drift-aware alternative [1][6]. PromptShift CRC embeds both prompts and model responses, then measures how far the current prompt stream has moved from the calibration pool [2]. It weights relevant or recent calibration examples more heavily and updates the risk level online after each observed violation [1][2]. The method also surfaces three diagnostics: realized risk error, prompt drift, and effective calibration size [1][2]. The authors provide conditions under which the approach controls risk up to terms that account for distribution mismatch and weighted quantile uncertainty [2]. In a synthetic prompt-shift benchmark, static conformal risk control broke down sharply after drift occurred, while PromptShift CRC delivered the best coverage among the adaptive baselines tested [1][2]. The same calibration layer was then evaluated on public benchmark-derived streams covering question answering, toxicity, summarization factuality, and long-context hallucination risk [1][2]. The work appears under arXiv’s Statistics > Machine Learning category and is accessible through the repository’s abstract page, which also links to community-built tools such as Bibliographic Explorer and Connected Papers via the arXivLabs framework [1][4][5]. arXivLabs, formalized in 2020, allows third-party collaborators to build experimental features on top of arXiv’s article pages while adhering to the platform’s values of openness, community, excellence, and user data privacy [4].