How LLMs See Creativity: Zero-Shot Scoring of Visual Creativity with Interpretable Reasoning

Six multimodal large language models can rate the creativity of AI-generated images and hand-drawn sketches at levels that substantially align with human judges, according to a study posted to arXiv. The models were tested without fine-tuning or exposure to example human ratings. The study evaluated Gemini 3 Flash, Gemma 4 31B IT, GPT-5.4 Mini, GLM-5v Turbo, Kimi K2.5, and Qwen 3.6 Plus on 992 AI-generated images and 1,500 hand-drawn sketches that had previously been scored for creativity by human raters [1]. The models produced ratings that correlated with human judgments at coefficients between .57 and .68 on the AI-generated set, and between .29 and .68 on the sketches [1]. Large language models are neural networks trained on vast text corpora for natural language processing tasks, and multimodal variants extend this capability to visual inputs [2]. The researchers sought to determine whether such models could serve as zero-shot judges of visual creativity, meaning they received no additional training or calibration examples before generating scores [1]. The paper also examined the step-by-step reasoning that three of the models produced alongside their ratings. The reasoning output revealed what visual features the models attended to and how they balanced originality against technical quality when justifying a score [1]. However, the presence of this reasoning did not improve the alignment between model ratings and human ratings [1]. The study arrives as AI companies continue to release models with expanding capabilities. DeepSeek, a Chinese AI firm, launched its R1 model in January 2025 with performance comparable to OpenAI's GPT-4, while reporting training costs far below those of its American rivals [8]. The arXiv paper's authors have released an open scoring application that implements their evaluation pipeline, making the method available for further testing [1]. arXiv, where the paper appeared, has integrated with Hugging Face Spaces to allow researchers to attach interactive demos directly to paper abstract pages, a collaboration designed to make machine learning research more accessible and reproducible [5][6].