Explain Like I'm 5 or Whatever I Choose: Evaluating the Interactive Potential of Language Model Responses

A new evaluation framework reveals that leading large language models struggle to reliably adjust the complexity of their responses on demand, a capability seen as essential for next-generation scientific search interfaces, according to a preprint published on arXiv [1]. The study, posted June 5, tested GPT-5.1, GPT-5 mini, Claude Sonnet 4.5, and DeepSeek-V3.1 by asking each to generate five responses at different levels of language complexity for 98 scientific queries [1]. The framework was built on a formative study with 16 participants and is inspired by direct manipulation interfaces from human-centered design literature, moving beyond the assumption of a single, static chat window [1]. While all models produced responses that varied in complexity, the changes were frequently misaligned with the requested direction. Claude Sonnet 4.5, the best-performing model, shifted reliable complexity measures correctly only 46% of the time [1]. The findings held when researchers increased the sample size and tested alternative complexity levels [1]. The work addresses a gap in how large language models are assessed for scientific information seeking. Current evaluations often involve live or multi-turn tests with real users but still assume a single, static chat interface [1]. As models are embedded into more dynamic interfaces, the authors argue, evaluations must incorporate interface-specific criteria such as the ability to modulate language along an interpretable axis [1]. This concept parallels the linguistic phenomenon of code-switching, where speakers shift between linguistic codes depending on social context or audience [3]. Large language models, which are neural networks trained on vast text corpora for generation and analysis tasks, form the backbone of modern chatbots [10]. Their outputs can be unreliable when training data is biased or inaccurate [10]. The new evaluation framework proposes a structured way to measure a model's interactive potential, treating language complexity as a controllable parameter rather than a fixed output characteristic [1]. The preprint was posted on arXiv, an open-access repository that hosts scientific papers in fields including computer science and has grown to receive about 24,000 submissions per month as of late 2024 [8].