MADE: Beyond Scoring via a Multilingual Agentic Diagnosing Engine for Fine-Grained Evaluation Insights

A new multilingual agentic system called MADE aims to convert the sprawling score tables of language-model benchmarks into fine-grained diagnostic reports, addressing what its creators describe as a landscape that is “metric-rich and insight-poor” [1]. The Multilingual Agentic Diagnosing Engine, detailed in a paper submitted to arXiv on 5 June 2026, decomposes post-evaluation analysis into five stages: planning, aggregate analysis, instance-level case inspection, multilingual and cultural reflection, and grounded report synthesis [1]. The system is paired with an expert-designed diagnostic set comprising 54 queries across 15 languages, and it was tested on an evaluation substrate spanning 33 model families, 11 benchmarks, 26 languages, 34 cultures, and 8.66 million evaluation records [1][2]. In experiments, MADE outperformed the strongest shared baseline by 47% in diagnosis report quality and was preferred by human multilingual experts in 87.9% of pairwise comparisons [1][3]. The engine scored 8.02 on a 7-dimensional judge, a gain of 2.57 over the baseline [3][4]. The authors argue that single large language models and open-ended agents are easily overwhelmed by the long, noisy diagnostic input that modern multilingual benchmarks produce, and that no reusable taxonomy previously existed for this task [2][4]. To address these challenges, MADE abstracts expert practice into a five-role workflow—Planner, Evidence Analyst, Case Analyst, Language Reflector, and Reporter—designed to pinpoint relevant evidence and guard against hallucination so that every claim remains grounded [3][4]. The diagnostic query set supplies the missing taxonomy, covering instance-, dataset-, and iteration-level needs [3]. When applied alongside human multilingual experts, MADE surfaced four actionable findings on deployment, version iteration, and cross-cultural pitfalls [1][2]. These findings redirect long-tail remediation away from single-score chasing and toward behaviour-stance auditing and slice-aware decisions, effectively turning benchmark score tables into model-selection and remediation guidance [3][4]. The work arrives as the broader research community increasingly pursues diagnostic evaluation frameworks that move beyond ranking. For instance, the MetaFine protocol decomposes robotic manipulation competency into understanding, perception, and controlled behavior through controlled semantic interventions and graded perturbations [5]. Similarly, the GEMA-Score system uses four specialized agents to assess clinical report quality across objective accuracy and subjective expressiveness, achieving a Kendall’s τ of 0.69 on one radiology benchmark [6]. MADE extends this diagnostic philosophy into the multilingual domain, where cultural nuance and linguistic diversity compound the difficulty of extracting actionable insight from raw scores.