Quantifying Prior Dominance in RAG Systems

Researchers have introduced the Normalized Context Utilization (NCU) metric to quantify contextual information gain in Retrieval-Augmented Generation (RAG) systems, revealing limitations in current evaluation methods and the dominance of parametric priors in large models^[1].

The NCU metric uses continuous token log-probabilities to measure contextual information extraction, distinguishing it from parametric memory recall. Evaluations of architectures ranging from 1.5B to 72B parameters showed that Small Language Models (SLMs) match or outperform larger models in strict factual extraction. A commercial API tested in the study overrode explicit external evidence in nearly half of adversarial conflicts and suffered from systemic confidence collapse when its parametric priors were contradicted^[1]. In related research, a cognitively-inspired framework called HippoRAG was found to fall between human and dense passage retrieval systems in interference sensitivity. Behavioral experiments and simulations validated the unified signal detection theory framework, with parameter recovery confirming the identifiability of its parameters. The study also found that a logarithmic specification of the framework was favored over a power-law alternative^[2].