DeFrame: Debiasing Large Language Models Against Framing Effects

A team of researchers has proposed a new method to reduce a subtle form of bias in large language models, showing that simply rewording a question can cause these systems to flip their answers on sensitive topics, even when standard fairness checks suggest they are unbiased. The study, published in the Findings of the Association for Computational Linguistics: ACL 2026, identifies "framing" as a key, underexplored contributor to hidden bias in large language models (LLMs) [1]. Framing refers to differences in how semantically equivalent prompts are expressed, such as asking "A is better than B" versus "B is worse than A" [1]. The authors, Kahee Lim, Soyeon Kim, and Steven Euijong Whang, found that fairness scores can vary significantly depending on how a prompt is phrased [3]. To quantify this problem, the researchers introduced the concept of "framing disparity" [1]. By augmenting standard fairness evaluation benchmarks with alternative framings, they demonstrated that LLMs often appear unbiased under one phrasing but produce biased responses under another [5]. This reveals a critical gap, as models that pass typical evaluations can still exhibit biased behavior in real-world applications where prompt wording is unpredictable [1]. The study further showed that existing debiasing methods, while improving overall fairness when averaged across framings, often fail to reduce these framing-induced disparities [1]. This means a model could be made fairer on average but remain highly sensitive to the specific words a user chooses, a vulnerability that could be exploited in disinformation attacks where strategic wording is used to manipulate audiences [10]. To address this, the team developed a new framework called DeFrame [4]. The approach is inspired by the dual-process theory from cognitive science, which distinguishes between fast, intuitive thinking (System 1) and slower, more deliberative reasoning (System 2) [4]. The researchers view framing-induced bias as a System 1 error, where the model reacts too intuitively to superficial wording cues [4]. DeFrame introduces a System 2-like step: the model is instructed to consider an alternative framing of the question, derive fairness guidelines from that perspective, and then revise its initial answer to be more consistent [4]. Experiments demonstrated that DeFrame reduces both overall bias and framing disparities, leading to responses that are fairer and more robust across different prompt phrasings [1]. The work addresses a dimension of model behavior that has been largely overlooked, ensuring more consistent outputs regardless of how a user formulates a query [5]. The research was conducted with eight different LLMs, confirming the approach's effectiveness across multiple architectures [9].