StylisticBias: A Few Human Visual Cues Drive Most Social Biases in MLLMs

A small set of human visual cues, including age and body type, drives the majority of social biases in multimodal large language models, according to a new controlled benchmark study that isolates how specific appearance attributes shift model judgments [1][2]. The study, posted to arXiv on June 18, introduces StylisticBias, a benchmark designed to measure attribute-level social bias in multimodal large language models, or MLLMs [1][2]. Researchers generated 500 photorealistic base faces and created roughly 50 single-attribute variations per face, producing about 25,000 images [1][2]. By keeping identity fixed and altering one visual attribute at a time, the design allows measurement of how specific cues shift model judgments across 25 binary social judgment scenarios evaluated on six MLLMs [2]. The findings show that age and body type dominate identity-level effects, while fashion style and other visual cues drive the largest attribute-level shifts [2]. Approximately 15 attributes account for nearly 80% of the total variation in social bias, indicating that bias is concentrated in a small set of visual cues [1][2]. Sensitivity is strongest in judgments semantically aligned with appearance, particularly socioeconomic and style-related judgments [1][2]. MLLMs are increasingly deployed in personally and societally consequential settings, yet the visual cues shaping how these models judge people have remained poorly understood [1][2]. Prior work often compared different individuals or groups, making it difficult to separate appearance effects from identity differences [2]. The StylisticBias benchmark addresses this gap by providing a controlled framework for fine-grained bias evaluation [2]. The researchers have released the code and dataset publicly on GitHub and Hugging Face, enabling other teams to replicate the evaluation and test their own models [1][2]. The release aligns with broader efforts to make machine learning research more accessible and reproducible. Hugging Face and arXiv have collaborated since 2022 to embed interactive demos directly alongside papers, allowing users to try models without writing code [3][4][5]. The study's lead author is Shaghayegh Kolli [1]. The work arrives as multimodal models gain wider deployment and scrutiny. Large language models, defined as machine learning models with many parameters trained on vast text corpora through self-supervised learning, have expanded rapidly into multimodal capabilities that process both text and images [7]. Companies such as DeepSeek have demonstrated that competitive models can be built at significantly lower cost than previously assumed, with its V3 model reportedly trained for $6 million compared to an estimated $100 million for OpenAI's GPT-4 [6].