AuAu: A Benchmark for Auditing Authoritarian Alignment in Large Language Models

A new benchmark called AuAu reveals that large language models from the United States, China, the European Union, and Russia can generate responses aligned with authoritarian attitudes, according to a study published this week [1]. The benchmark, detailed in a paper on arXiv, combines three evaluation methods: psychometric questions drawn from 15 human-validated instruments, contextual vignettes that probe intended actions in concrete situations, and responses to realistic user prompts [2]. Unlike earlier audits, AuAu measures not just a general proximity to authoritarianism but also the sub-concepts of Authoritarian Aggression, Authoritarian Submission, and Conventionalism [2]. Researchers tested 17 models originating from China, the EU, Russia, and the USA [1]. All models exhibited substantial authoritarian response rates under the psychometric evaluation, though those rates dropped significantly when the models were subjected to more realistic downstream tasks [2]. The study also found that introducing an authoritarian system prompt easily manipulated 15 of the 17 models, causing them to promote increased authoritarianism in their outputs [2]. The findings arrive as AI assistants become more deeply embedded in daily digital life. Meta, for instance, has rolled out its Meta AI assistant built on the Llama family of models across Facebook and WhatsApp [5]. Chinese firm DeepSeek launched its eponymous chatbot alongside the DeepSeek-R1 model in January 2025, offering performance comparable to OpenAI's GPT-4 at a fraction of the reported training cost [4]. The paper's authors argue that the results underscore the need for continued, systematic auditing of LLM-based AI systems to detect and mitigate undesired authoritarian tendencies [2]. The benchmark's code and data have been made publicly available on GitHub [2]. The research contributes to a growing body of work examining how the political and cultural contexts in which models are developed may shape their outputs, though the study does not draw direct causal links between a model's country of origin and its performance on the benchmark [1].