UXBench: Measuring the Actionability of LLM-Generated UX Critiques

A new benchmark called UXBench aims to measure whether large language models can produce reliable and actionable critiques of user interfaces, addressing a gap in evaluating these increasingly deployed AI judges. [1] Large language models are being used to inspect interfaces, diagnose usability problems, and propose repairs, but no controlled benchmark has existed to measure the reliability and actionability of their output across different product surfaces. [1] The UXBench framework, detailed in a paper submitted in 2026, provides a structured method for this evaluation. [1] The benchmark is built on local-first runnable web fixtures that cover ten product-surface families. [1] It uses coverage-gated browser exploration, which requires models to gather interaction evidence before generating a report. [1] Each model produces a structured UX report assessed across seven rubric dimensions. [1] The quality of these reports is measured by whether a fixed downstream repair agent can improve the interface based on the critique. [1] Researchers evaluated eight frontier models using an automated repair-lift protocol and a blind human validation study. [1] The findings indicate that UX judging is not a saturated or one-dimensional capability. Models showed meaningful differences in report actionability, distinct rubric-level repair signatures, and varied in their reliability across different fixtures. [1] Leadership among the models also traded off depending on the surface category being evaluated. [1] The development of specialized benchmarks like UXBench reflects a broader trend in machine learning research, where new datasets are created to test specific model capabilities and determine if they complement existing resources. [4] This is considered important for consolidating data from various computational methods and for techniques like transfer learning, where a model trained on a larger dataset can be fine-tuned for a smaller, related task. [4] The UXBench paper was shared on the arXiv preprint server, which supports community-developed tools through its arXivLabs framework for sharing new features. [1]