LazyAttention: Efficient Retrieval-Augmented Generation with Deferred Positional Encoding

Researchers have introduced two new models, LazyAttention and GraphER, aimed at improving the efficiency of retrieval-augmented generation in large language models.

LazyAttention is a novel attention mechanism that enables zero-copy, position-agnostic KV reuse by kernelizing deferred positional encoding^[1]. This allows a single physical KV copy to serve multiple logical requests at arbitrary positions. According to the study, LazyAttention reduces time-to-first-token (TTFT) by 1.37 times and increases inference throughput by 1.40 times compared to Block-Attention, while maintaining comparable output quality^[1]. Meanwhile, GraphER is a retrieval method that leverages the organizational structure of data to capture proximity relationships beyond semantic similarity^[2]. It constructs a graph at query time based on these proximities and applies graph-based ranking to surface top candidate documents. GraphER requires no additional graph infrastructure and integrates seamlessly with standard vector stores, making it retriever-agnostic and supportive of multiple forms of proximity^[2]. The GraphER submission was made on March 26, 2026, with a revision on June 8, 2026.