Causal Preference Elicitation

Researchers have proposed a Bayesian framework for expert-in-the-loop causal discovery and developed a method to separate the effects of self-consistency elicitation and reward design in reinforcement learning from verifiable rewards (RLVR).

The Bayesian framework, called causal preference elicitation, actively queries local edge relations to concentrate a posterior over directed acyclic graphs (DAGs)^[1]. It models noisy expert judgments with a three-way likelihood over edge existence and direction. Posterior inference uses a flexible particle approximation, and queries are selected by an efficient expected information gain criterion on the expert's categorical response. In a separate development, researchers found that the common method of estimating the reward-design effect in RLVR is systematically biased, conflating self-consistency elicitation with genuine reward-design signal^[2]. A pre-registered 2x2x2 factorial study confirmed non-additivity of the effects of prior strength, alignment, and crossover. The reward-design fraction of the naive estimator ranged from 0.139 at weak prior to 0.05 at strong prior^[2].