Optimizing Abstractive Summarization With Fine-Tuned PEGASUS

A fine-tuned PEGASUS model has achieved state-of-the-art results on the XL-Sum English corpus for abstractive summarization, outperforming a baseline mT5 model across multiple ROUGE metrics, according to a paper posted on arXiv [1]. The work, submitted by Sadiul Arefin Rafi on June 24, 2026, focuses on abstractive summarization, a technique that generates concise summaries capturing the main ideas of a source text without simply extracting sentences [1][2]. Transformer-based architectures such as BART, T5, and PEGASUS have made this process more efficient and accurate [2]. The objective of the paper was to fine-tune PEGASUS on the XL-Sum English corpus to surpass the performance of a baseline mT5 model [1][2]. Evaluation relied on the ROUGE metric, which compares automatically generated summaries against human-created references [1][2]. According to the paper, the fine-tuned PEGASUS model delivered a 4.04% improvement in the ROUGE-1 score, a 15.25% increase in the ROUGE-2 score, and a 3.39% improvement in the ROUGE-L score relative to the baseline [1][2]. The authors state that, to the best of their knowledge, these results represent state-of-the-art performance on the XL-Sum English Corpus [1][2]. The paper was posted on arXiv, an open-access repository for electronic preprints that, as of late 2024, receives about 24,000 submissions per month and has hosted over two million articles since its launch in 1991 [6]. arXiv provides a framework called arXivLabs that allows community collaborators to develop experimental tools, such as bibliographic explorers and code finders, which appear on article record pages [4][5]. The repository is moderated but does not conduct peer review [6].