When Should LLMs Be Less Specific? Selective Abstraction for Reliable Long-Form Text Generation

Researchers have proposed a framework called Selective Abstraction to improve the reliability of long-form text generation by large language models (LLMs) by trading specificity for reliability.

LLMs are prone to factual errors that erode user trust and limit adoption in high-risk settings^[1]. To mitigate this risk, the Selective Abstraction framework enables LLMs to selectively reduce the detail of uncertain content. The framework decomposes responses into atomic claims and replaces uncertain atoms with higher confidence, less specific abstractions^[1]. Researchers evaluated the framework on six open-source models using the FactScore and LongFact-Objects benchmarks, finding that it improved the area under the risk-coverage curve (AURC) by up to 27.73% over claim removal^[1]. Meanwhile, a separate study introduced PersistBench to measure safety risks of long-term memories in LLMs, revealing a high failure rate among 18 frontier and open-source LLMs tested, with a median failure rate of 53% on cross-domain samples and 97% on sycophancy samples^[2]. The integration of long-term memory with LLMs is becoming more prevalent in conversational assistants to enhance personalization, but this persistence can introduce safety risks such as cross-domain leakage and memory-induced sycophancy^[2].