Beyond Text Following: Repairable Arbitration Reversals in Audio-Language Models

Audio-language models routinely override clear audio evidence when faced with conflicting text, but a new training-free decoding method can correct the behavior without retraining, according to a study posted to arXiv on June 3, 2026 [1]. The study examined five audio-language models across four conflict tasks and found that in 64.1% of conflict samples, the model’s same-audio branch preferred the audio-supported answer while the joint audio-text branch preferred the text-supported answer [1]. This pattern, which the authors term “repairable arbitration reversal,” indicates that the relevant audio evidence is encoded but loses when the model arbitrates between modalities [1]. Activation patching localized the reversal to answer-position computation, and the patch-induced direction tracked the score difference between the audio-supported and joint answers with a Spearman rank correlation of 0.93 [1]. Prior work has documented a similar asymmetry. A separate study of Gemini 2.0 Flash found that when two conflicting text sources were presented with explicit reliability cues, the model followed the unreliable text only 1.6% of the time. But when one source was audio rather than text, with identical reliability framing, the text-dominance rate rose to 16.6% — a tenfold increase [5]. That study proposed the gap reflects a distinction between information content and arbitration accessibility: audio embeddings preserve more task-relevant signal, but text is easier for the model to reason over when modalities conflict [5]. Using the two-branch diagnostic, the authors of the new paper propose Gated Audio Counterfactual Logit Correction, or GACL, a decoding rule that interpolates between joint and same-audio reference scores, gated by branch disagreement and reference reliability [1]. Under a strict 5-percentage-point faithfulness-drop budget, GACL improved conflict audio-following accuracy by 17.8 normalized AUC points over the best contrastive decoding baseline [1]. The method transferred without retuning to vision-text arbitration on the MC2 benchmark, raising adversarial accuracy by up to 40.5 percentage points, which the authors say suggests the counterfactual design generalizes beyond audio-text settings [1]. The work was released through arXivLabs, a framework for community collaborators to develop and share features on the arXiv platform, and code and data were made available through Hugging Face [1].