Detecting Hallucinations for Large Language Model-based Knowledge Graph Reasoning

A research team has introduced LUCID, a method designed to detect hallucinations in large language models when they perform knowledge graph reasoning, an approach that infers new facts from existing structured data [1]. Knowledge graph reasoning is used in question answering, recommendation systems, and decision support [2]. Large language models, or LLMs, have become popular tools for this task because they can incorporate retrieved knowledge graph information [1]. However, these models remain prone to generating factually incorrect outputs, a phenomenon known as hallucination, which can spread misinformation and lead to unreliable decisions [1]. Existing detection methods typically examine a model’s internal states or check whether outputs are consistent with retrieved text, but they ignore the structural relationships within a knowledge graph [1]. LUCID addresses this gap by jointly analyzing LLM attention scores, the semantic meaning of graph elements, and the graph’s structure [1]. The system extracts node and edge features from attention scores and semantic similarities, then processes them through a graph neural network [2]. The researchers constructed manually annotated benchmark datasets to test the method [1]. Across nine datasets, LUCID outperformed 15 baseline approaches [1]. The work was posted on arXiv, an open-access repository for scientific preprints that has hosted over two million articles and receives about 24,000 submissions per month as of late 2024 [10]. Machine learning, the broader field encompassing this work, relies on statistical algorithms that learn from data to perform tasks without explicit programming [3]. Since the 2020s, generative AI systems built on these principles have become widely available, producing text, images, and audio from prompts [5]. The rapid expansion of such tools has intensified scrutiny of their reliability, including the hallucination problem that LUCID targets [1][5].