Small edits, large models: How Wikipedia advocacy shapes LLM values

A coordinated group of Wikipedia editors has demonstrated that a small number of targeted edits can measurably influence how large language models discuss animal welfare, according to a new study [1]. The study, posted to arXiv, examines the Pro-Animal Wikipedians (PAW), a group of advocates who add sourced animal welfare content to relevant articles. The group made 125 edits across 115 pages [1]. Wikipedia appears in nearly every major language model training dataset and is weighted more heavily than general web-crawled text, the researchers note [1]. Using gradient-based data attribution methods on Meta's Llama 3.1 8B model, the researchers found that PAW-edited sections constituted 68 percent of the highest-attributed documents for animal welfare queries, a result with a p-value below 0.0001 [1]. For unrelated queries about the same companies, the figure was 52 percent, with a p-value of 0.53, indicating the model links PAW content specifically to animal welfare topics rather than to the entities in general [1]. A second method, MAGIC counterfactual influence estimation run on Llama-3.2-1B across five random training-order seeds, produced a sharper result: in every seed, all 10 of the top-10 most influential documents on animal welfare queries were PAW edits, while on general queries the same top-10 sat at chance, between 4 and 6 of 10 [1]. Mean PAW influence exceeded mean control influence on animal welfare queries with p < 0.0001 in every seed, an effect 6 to 30 times larger than on general queries [1]. The researchers also fine-tuned separate models on PAW content and control content. The PAW-trained model reduced perplexity on animal welfare text from 12.4 to 8.4, while the control-trained model reduced perplexity on control text from 16.1 to 11.4 [1]. Leave-subset-out validation yielded a Spearman rho of 1.00 across all 10 runs [1]. The findings add to a body of academic research on Wikipedia's influence. Studies have examined the encyclopedia's reliability, systemic biases, and its role as a dataset for machine learning [9]. Wikipedia is the largest reference work in history, with the English edition containing over 7 million articles and attracting more than 1.5 billion unique device visits per month as of April 2024 [8]. The study also surfaces broader questions about how coordinated editing campaigns intersect with the platform's open-collaboration model. Public relations practice, as defined on Wikipedia, involves managing and disseminating information to influence public perception, a process that is controlled internally by an organization [3]. The PAW campaign, while volunteer-driven, operates with a similar informational objective: shaping how a specific topic is presented in a source that feeds AI training pipelines. Wikipedia has long grappled with systemic bias. A 2018 survey across 12 language versions found that 90 percent of respondents identified as male, and the platform has been criticized for gender and geographical bias [7][8]. The new research suggests that even small editorial groups can leave a detectable imprint on downstream AI behavior, raising the stakes for content governance on the platform.