Breaking the Mirror: Activation-Based Mitigation of Self-Preference in LLM Evaluators

Researchers have identified a 'self-preference bias' in large language models (LLMs) used as automated evaluators, where they favor their own outputs over others, potentially undermining fairness and reliability.

The bias is particularly problematic for tasks like preference tuning and model routing, according to a study published on arXiv^[1]. Researchers found that LLMs tend to favor their own outputs when acting as judges, a behavior that may be due to 'narcissism or experimental confounds'^[2]. To mitigate this issue, the researchers introduced a curated dataset and constructed steering vectors using two methods: Contrastive Activation Addition (CAA) and an optimization-based approach. Their results showed that steering vectors can reduce unjustified self-preference bias by up to 97%^[1]. However, the researchers also noted that steering vectors are unstable on legitimate self-preference and unbiased agreement. In a related study, researchers corrected for evaluator quality by comparing voting distributions between self-evaluations and evaluations of other models, finding that only 51% of examples in previous findings retained statistical significance against this null hypothesis^[2].