MedBench v5: A Dynamic, Process-Oriented, and Hallucination-Aware Benchmark for Clinical Multimodal Models

A new benchmark for clinical artificial intelligence models, MedBench v5, shifts evaluation from static question-answering to dynamic, process-oriented testing that tracks how models reason and where they hallucinate, according to a preprint posted to arXiv on June 23, 2026 [1]. The framework introduces a dual-dimensional structure: Clinical Cognitive Responsiveness, spanning 14 sub-dimensions, and Medical Atomic Skills across four agent environments, together covering 63 tasks [1][2]. Three switchable information-flow stressors — omission, contradiction, and evidence delay — allow researchers to conduct factorized degradation analysis, isolating specific failure modes [1][2]. A dynamic process audit protocol with five reasoning nodes generates model-specific failure fingerprints, while a hallucination propagation monitor tracks errors across initiation, propagation, anchoring, and contradiction interaction, capturing what the authors term “silent hallucination” [1][2]. Experiments on frontier models revealed that strong overall task performance does not guarantee process stability. Stressors mainly disrupted contradiction detection, diagnosis updating, hallucination propagation, and contradiction-based self-correction, even when final evidence grounding appeared superficially stable [1][2]. The findings suggest that models can produce correct answers while their internal reasoning chains remain brittle under information-flow perturbations [1][2]. The paper arrives as the volume of AI preprints on arXiv continues to grow. The repository, founded in 1991, now receives roughly 24,000 submissions per month and surpassed two million articles by the end of 2021 [4]. Preprints on arXiv are moderated but not peer-reviewed, meaning the MedBench v5 framework has not yet undergone formal journal scrutiny [4]. Separate research on large language model methodology, also posted to arXiv, found that when LLMs are prompted with only a research question, they suggest a much narrower range of methods than researchers actually use. The effective number of model entities contracted from 1,232 to between 59 and 96, and inter-LLM rank correlations exceeded LLM-to-paper correlations, indicating that the distortions are largely shared across models [3]. That study concluded that researchers who rely on LLM suggestions without cross-checking risk narrowing their methodological search space toward a more concentrated default [3]. The finding parallels the MedBench v5 authors’ emphasis on process auditing: both lines of work highlight that surface-level outputs can mask underlying brittleness or homogeneity [1][3]. MedBench v5 provides a unified infrastructure for capability profiling, controllable stress testing, process auditing, and hallucination trajectory analysis in clinical AI evaluation [1][2]. The benchmark is designed for language, vision-language, and agent-based clinical multimodal models [1][2].