Prompt, Plan, Extract: Zero-Shot Agentic LLMs Workflows for Lung Pathology Extraction from Clinical Narratives

A zero-shot agentic workflow built on open-source large language models can extract lung pathology information from clinical narratives with performance approaching a fully supervised system, according to research posted to arXiv on June 18, 2026 [1][2]. The study evaluated five open-source generative LLMs on their ability to populate 13 College of American Pathologists synoptic fields from lung resection pathology reports [1][2]. The best-performing zero-shot model, GPT-OSS-20B, achieved a Micro-F1 score of 0.893 and recall of 0.949 [1][2]. The state-of-the-art supervised GatorTron Named Entity Recognition and Relation Extraction baseline reached a Micro-F1 of 0.960 [1][2]. The authors note that the zero-shot workflow accurately extracted complex relations, including Pathologic Stage, without any task-specific training [1][2]. Information extraction from pathology reports is critical for cancer staging and populating tumor registries, yet much of the relevant data remains locked in narrative text, making manual abstraction both labor-intensive and error-prone [1][2]. Traditional supervised NLP pipelines require costly manual annotation and can suffer cascading failures when upstream entities are missed [1][2]. The agentic workflow described in the paper offers an alternative that avoids these annotation costs entirely [1][2]. The paper appeared on arXiv, an open-access repository that hosts electronic preprints across disciplines including computer science, mathematics, and quantitative biology [6]. Founded in 1991, arXiv passed the two-million-article milestone by the end of 2021 and now receives roughly 24,000 submissions per month [6]. Submissions are moderated but not peer-reviewed [6]. The platform also supports community-developed tools through its arXivLabs framework, which allows collaborators to build features such as citation explorers and code-finding services directly on article pages [4][5]. Large language models, the class of machine learning model used in the study, are trained with self-supervised learning on vast text corpora and contain many parameters [8]. The GPT-OSS-20B model tested in the research is one such open-source system [1][2]. The authors conclude that open-source, zero-shot agentic LLMs represent a low-cost solution for extracting lung pathology information from clinical narratives [1][2].