Evaluating Second-Order Bias of LLMs Through Epistemic Entitlement

A new study from researchers at the University of Toronto argues that current evaluations of social bias in large language models miss a critical dimension: how the models judge biased content, rather than just what they generate. The paper, posted to the arXiv preprint server on June 16, 2026, introduces the concept of "second-order bias" to capture this oversight [1][2]. The work, led by Ramaravind Kommiya Mothilal, contends that as LLMs are increasingly deployed as automated judges of bias, they can exhibit social biases in subtler ways that existing methods do not systematically capture [1][2]. The authors term this phenomenon "second-order bias," defined as social bias in an LLM's judgment about social bias itself [2]. The paper draws on entitlement epistemology, a philosophical framework, to conceptualize bias as misplaced foundational knowledge that shapes an agent's rational inquiry [2]. From this, the team derived a logical reasoning task in which LLMs must judge to whom a biased text is acceptable or non-acceptable [2]. The researchers developed two simple metrics to quantify the problem: one measuring how biased LLM judges are in inferring demographics for acceptability without sufficient support, and another assessing how these inferences vary across groups targeted by biased texts [1][2]. When evaluating both open and closed models, the study found that the task evaded safety guardrails by surfacing bias in model judgment [2]. The bias varied systematically across target groups, reflected implicit social maps, and showed that models remain triggered by demographic labels [2]. The paper was submitted as a 109 KB preprint to arXiv, an open-access repository for scientific papers in fields including computer science that, as of late 2024, receives about 24,000 articles per month [1][6]. The repository is not peer-reviewed but provides rapid dissemination of research findings [6]. The authors have released their code and model responses on GitHub [2]. The study points to the need for LLM bias evaluation in judgment tasks and for more theoretically grounded approaches to bias evaluation in natural language processing broadly [2].