The Register Gap: A Meaning Intelligence Framework for Nigerian Public Discourse

Researchers have proposed a new evaluation framework for artificial intelligence systems processing Nigerian public discourse, arguing that existing benchmarks fail because they mistake translation problems for failures of cultural and situational context. The Meaning Intelligence Framework (MIF), detailed in a paper submitted to arXiv on 18 June 2026, introduces a nine-dimension annotation schema designed to separate surface sentiment from a speaker's true communicative intent [1][2]. The authors contend that current benchmarks for Nigerian languages, such as NaijaSenti and AfriSenti, reduce sentiment classification to a simple three-way polarity task — positive, negative, or neutral — and in doing so miss the dominant failure mode of AI systems: context failure [1][2]. The same utterance, they note, can carry opposite pragmatic force depending on speaker, audience, and situation [2]. The MIF operationalises this insight across nine scored dimensions: register, surface sentiment, true intent, irony, coded subtext, risk tier, annotator confidence, speaker emotion, and recommended communications action [1][2]. To test the framework, the team constructed a 30-item calibration dataset spanning Standard English, Nigerian English, Nigerian Pidgin, and code-mixed registers, and evaluated the frontier language model Gemini 2.5 Flash under both zero-shot and schema-informed prompting conditions [1][2]. The headline finding is what the authors term the “Register Gap.” Under zero-shot conditions, the model achieved a register classification accuracy of 33.3 percent. When provided with the MIF schema in-context, accuracy rose to 73.3 percent, a jump of 40 percentage points [1][2]. The composite Meaning Intelligence Score increased by 5.4 points, from 73.2 to 78.6, with the largest practical gains recorded in register identification, coded-subtext detection — which improved by 10 points — and strategic action recommendation, which rose by 10.3 points [1][2]. The paper also reports that model capability and cultural competence appear decoupled. GPT-5 and Gemini 2.5 Pro scored lower composite Meaning Intelligence Scores than Gemini 2.5 Flash, and neither benefited from schema-informed prompting [2]. The researchers have released the framework specification, annotation guidelines, and the 30-item public calibration set to support reproducibility, while retaining a private holdout corpus for contamination-protected evaluation [1][2]. The work arrives as the broader AI community grapples with how large language models handle culturally specific communication. Large language models are machine learning systems trained on vast amounts of text for natural language processing tasks [10]. The MIF paper is hosted on arXiv, a platform that has integrated with Hugging Face Spaces to allow researchers to attach interactive demos to their papers, enabling wider audiences to explore model behaviour without writing code [6][7][8].