Characterizing Cultural Localization in AI-Generated Stories

A new study finds that when AI models generate stories for different nationalities, only 9 to 17 percent of the vocabulary distinguishes one culture from another, while the underlying narrative remains largely identical, suggesting a shallow form of localization that relies on surface-level cultural markers [1]. The research, posted to arXiv, proposes a method to measure whether AI-generated stories achieve cultural localization through what the authors call “templated localization” — swapping in names, foods, and locations atop a generic narrative — or through “holistic localization,” which would vary plots, values, and themes [1]. The team prompted five models to produce stories across 125 topics for 193 nationalities, then identified the lexical tokens that set each nationality’s output apart [2]. After stripping those tokens, they measured the similarity of the remaining text. The result: only 9 to 17 percent of the vocabulary accounted for cross-national variation, and the stripped-down narratives contained repeated multi-word sequences, pointing to a shared, culturally agnostic template [3]. “Our method reveals that localization primarily occurs through surface-level lexical differences, suggesting that stories may use a homogeneous underlying narrative,” the authors write [3]. The similarity among the template images was higher than among the original stories, and masking an equivalent number of random words produced lower similarity, confirming that the cultural markers themselves were doing the distinguishing work [3]. The study further characterized those markers for stereotypicality and offensiveness. Markers from 19 countries, mostly in the Global South — predominantly in Africa and West Asia, with lower-resourced dominant languages — were on average offensive [2][3]. The finding aligns with broader concerns about cultural representation in language models. A separate evaluation of six popular LLMs generating stories about Indian cultural identities found that 88 percent of the outputs contained misrepresentations, with errors more frequent in mid- and low-resourced languages and in peri-urban regions [5]. That study, which produced a taxonomy of seven categories of cultural misrepresentation, noted that social practices, social norms, and food were consistently mishandled [5]. Other recent work has approached the problem from a mechanistic angle, using sparse autoencoders to identify interpretable features that encode culturally salient information. Those researchers found that under underspecified prompts, models defaulted to Anglophone cultures roughly 60 percent of the time, with 33.1 percent of responses aligning most closely with the United States and 26.9 percent with the United Kingdom [4]. Steering those internal features improved cultural faithfulness and surfaced rarer cultural concepts, suggesting that models possess long-tail knowledge that standard prompting fails to elicit [4]. The concept of adapting content across cultures is not new. The translation industry has long practiced transcreation — a portmanteau of translation and creation — which aims to preserve intent, style, tone, and context while adapting not only words but also images and video for the target audience [8]. The new arXiv paper provides a quantitative lens on how far current AI systems fall short of that standard, reducing cultural difference to a thin lexical veneer over a uniform narrative core [1][3].