ProvenanceGuard: Source-Aware Factuality Verification for MCP-Based LLM Agents

Researchers have introduced ProvenanceGuard, a verification system designed to catch a specific failure in AI agents that use the Model Context Protocol: answers that are factually supported but attributed to the wrong source, a problem the team calls cross-source conflation [1][2]. Large language model agents increasingly rely on the Model Context Protocol, or MCP, to pull answers from disparate evidence sources such as search engines, APIs, databases, and clinical records [2]. Standard factuality checks typically assess whether an answer is supported by the pooled body of evidence, but they overlook a provenance-sensitive error where a claim is supported somewhere in the evidence set while being credited to an incorrect source [1][2]. The new verifier, detailed in a paper posted to arXiv on June 16, 2026, is designed to flag this exact failure mode [1]. ProvenanceGuard works by consuming captured MCP traces that include stable tool IDs, source IDs, and raw outputs. It breaks an agent’s answer into atomic claims, routes each claim to its source-specific evidence, and checks support using natural language inference and a token-alignment proxy. The system then compares the stated attribution with the routed source and issues per-claim verdicts along with an answer-level allow or block decision [1][2]. Answers that are blocked can be repaired through retrieval-augmented revision and re-verified [2]. The researchers evaluated the system on 281 medical-domain MCP-agent traces. From an adjudicated subset of 266 traces, they produced 2,325 LLM-assisted claim labels, with 361 held-out labels verified by humans [1][2]. On a 40-trace held-out split, ProvenanceGuard achieved a block F1 score of 0.802 and source accuracy of 0.858 across 260 source-eligible claims, outperforming source-blind baselines that do not emit claim-to-source IDs [1][2]. On a harder multi-source benchmark, block F1 reached 0.846, while source-plus-relation accuracy fell to 0.229, indicating that pinpointing exact source ownership remains difficult when sources are semantically close [1][2]. In a set of 50 controlled clinical conflation probes, the system detected every injected attribution swap and retained no wrong attribution [1][2]. The paper’s authors note that source attribution constitutes an independent axis for factuality verification in MCP-based agents, separate from traditional support checks [2]. The work appears on arXiv, an open-access repository that has hosted more than two million e-prints since its launch in 1991 and currently receives about 24,000 submissions per month [6].