CREDENCE: Claim Reduction for Decomposition & Enhanced Credibility -- Semantic Metrics and Convergence Analysis

A new framework called Credence aims to improve the reliability of automated fact-checking by fixing flaws in how compound claims are broken down and evaluated, according to a paper published on arXiv [1]. The framework, presented by Huu Vu Phuong Tran, addresses two key shortcomings in prior work: the use of token-overlap metrics that underestimate decomposition quality and the absence of formal termination analysis for the repair loop [1][2]. Credence introduces Semantic-F1, a metric that uses BGE-large cosine similarity to measure fidelity, which the authors state resolves the penalisation issues of Jaccard-based metrics and improves downstream fact-checking accuracy [2]. Experiments across three evaluation benchmarks—SocialClaimSplit, WikiSplitBench, and ClaimDecompBench—showed that Semantic-F1 outperformed Jaccard-F1 by 15 to 32 percentage points [2]. The benchmarks span social-media, encyclopaedic, and news domains to measure cross-domain generalisation [2]. The Exact-Preservation Ratio (EPR) ranged from 0.94 to 1.00 on SocialClaimSplit and WikiSplitBench, while ClaimDecompBench included lower base EPR cases down to 0.824 due to harder news-domain constructions [2]. The paper also provides convergence theorems that formally characterise four properties of the repair pipeline. Rule-based repair is shown to be monotone and finitely terminating under an oracle parser assumption, while LLM-based self-repair is provably non-monotone and requires an early-exit guard [2]. Rule-repair reduced the Atomicity Violation Rate by 47 to 100 percent relative to the base model without degrading fidelity [2]. The research was benchmarked across four decomposer models ranging from 3.8 billion to 12 billion parameters, along with a closed API model [2]. Large language models, which are trained with self-supervised learning on vast amounts of text, underpin many such decomposer systems [7]. The paper appears on arXiv, a preprint repository that has integrated with Hugging Face Spaces to allow community-built demos to appear alongside papers, increasing reproducibility and accessibility of machine learning research [3][4][5].