Building Reliable Long-Form Generation via Hallucination Rejection Sampling

Researchers have proposed two novel methods to improve the performance of large language models and reduce their tendency to hallucinate incorrect or unsupported content.

The first method, called Segment-wise HAllucination Rejection Sampling (SHARS), uses a hallucination detector to identify and reject hallucinated segments during generation and resample until faithful content is produced^[1]. By retaining only confident information and building subsequent generations upon it, SHARS mitigates hallucination accumulation and enhances factual consistency. Empirical evaluations demonstrate that SHARS substantially reduces hallucinations in long-form generation while preserving or even improving the informativeness of generation. Meanwhile, a second approach, Constrained Adaptive Rejection Sampling (CARS), improves the sample-efficiency of rejection sampling without distributional distortion. CARS begins with unconstrained language model sampling and adaptively rules out constraint-violating continuations by using a trie to record these continuations and subtract their probability mass from future draws^[2]. CARS achieves higher efficiency and stronger sample diversity than greedy constrained decoding methods and methods that approximate the language model's distribution.