A Self Consistency Based Reranking for Narrative Question Answering

A new inference framework for narrative question answering uses multiple candidate answers and semantic agreement to select a final response, improving robustness without altering the underlying model architecture, according to a preprint posted to arXiv on 14 June 2026 [1][2]. Narrative question answering requires models to parse long texts and track relationships across events, a task where single-decoding outputs often produce inconsistent answers [1][2]. The proposed self-ensemble reranking method addresses this by generating several candidate answers for each story-question pair and then selecting the output with the highest semantic consensus among the group [1][2]. The approach combines pretrained and fine-tuned language generation with multi-answer inference and similarity-based reranking [1][2]. Researchers evaluated the framework on the NarrativeQA dataset using FLAN-T5 Base, FLAN-T5 Small, and Pegasus-Large, testing both baseline and fine-tuned configurations [1][2]. FLAN-T5-Base recorded the strongest overall result, moving from a baseline of 82.32% to 86.66%, a gain of 4.34 percentage points [1][2]. The largest single improvement came from Pegasus-Large, which climbed from 72.50% to 87.07%, an increase of 14.57 percentage points [1][2]. The preprint appeared on arXiv, an open-access repository that hosts electronic preprints across physics, mathematics, computer science, and related fields [6]. Founded in 1991, the repository passed two million articles by the end of 2021 and currently receives about 24,000 submissions per month [6]. Papers on arXiv are moderated but not peer-reviewed [6]. The site also supports community-built tools through arXivLabs, a framework that lets third-party developers create experimental features such as citation explorers and recommender systems that appear on article pages [4][5]. Large language models, the broader category that includes FLAN-T5 and Pegasus, are machine learning systems with many parameters trained on extensive text corpora for tasks such as language generation [8]. The self-ensemble method does not require changes to these models' internal architectures, relying instead on inference-time sampling and reranking [1][2].