LLM-Based Visual Explanation Evaluation Framework for Assessing the Explainability of Facial Skin Disease Classification Models

A new evaluation framework uses large language models to judge how well AI vision models explain their diagnoses of facial skin diseases, according to a paper submitted on 15 Jun 2026 [1]. The framework targets a known gap: while many studies have boosted classification accuracy through data augmentation, fewer have systematically checked whether the models focus on clinically relevant lesion regions [2]. The authors applied geometric, color-based, and mixed augmentation strategies to three convolutional neural networks — EfficientNet-B0, MobileNetV3, and ResNet18 — and used Grad-CAM to generate heatmap-style visual explanations of each model’s decision [2]. To evaluate those explanations, the researchers built an “LLM-as-a-Judge” system powered by GPT-5.5, Gemini 3.5 Flash, and Claude Sonnet 4.6 [2]. The LLMs scored the Grad-CAM outputs on two axes: how well the highlighted areas matched actual lesion locations, and the overall trustworthiness of the explanation [2]. A progressive prompt-engineering strategy supplied the LLMs with evaluation rubrics, clinical knowledge, penalty rules, and structured output formats to improve consistency and clinical grounding [2]. Artificial intelligence has undergone rapid change since deep learning began outperforming earlier techniques around 2012, and the 2017 transformer architecture accelerated progress further [3]. The 2020s brought an AI boom driven by generative models that can produce images, audio, and video from text prompts [3]. The new framework sits at the intersection of those trends, applying generative LLMs not to create content but to audit the reasoning of other AI systems. The work appears as a preprint on arXiv and has not yet been peer-reviewed [1]. The authors frame it as a step toward making dermatology AI more transparent, arguing that high accuracy alone is insufficient if clinicians cannot see whether a model is looking at the right tissue [2]. No external validation on clinical outcomes is reported in the preprint [1].