Improving Scientific Document Retrieval with Academic Concept Index

A new method for improving scientific document retrieval uses an academic concept index to address the vocabulary and annotation gaps that limit general-domain search tools in scholarly settings, according to a paper posted to arXiv. The paper, submitted on 2 January 2026 and revised on 17 June 2026, notes that adapting general-purpose information retrieval systems to scientific domains is difficult because of scarce domain-specific relevance labels and a mismatch in terminology and information needs [1][2]. Information retrieval, broadly, is the task of identifying and retrieving system resources relevant to an information need, with web search engines being the most visible application [3]. Full-text search techniques, which examine all words in stored documents, have been in use since the 1960s [4]. Recent approaches have used large language models, or LLMs, to generate synthetic queries for fine-tuning and to produce auxiliary contexts that support relevance matching [2][6]. LLMs are neural networks trained on vast text corpora for tasks such as generation and summarization [6]. However, the authors argue that both strategies overlook the diverse academic concepts within scientific papers, often yielding queries and contexts that are redundant or conceptually narrow [2]. To address this, the researchers introduce an academic concept index that extracts key concepts from papers and organizes them using an academic taxonomy [2]. Metadata, which describes characteristics of data such as a document’s author or keywords, has long been used to help users find relevant information [5]. The new index builds on that principle by structuring concepts to guide two enhancements. The first, called CCQGen, conditions LLMs on uncovered concepts to generate complementary queries with broader concept coverage. The second, CCExpand, uses document snippets as concise responses to those concept-aware queries [2]. Experiments reported in the paper indicate that incorporating the academic concept index into both query generation and context augmentation produces higher-quality queries, better conceptual alignment, and improved retrieval performance [2]. The work was led by SeongKu Kang and appears under the Computer Science > Information Retrieval category on arXiv [1].