PVminerLLM2: Improving Structured Extraction of Patient Voice via Preference Optimization

A research team has introduced PVminerLLM2, a set of language models designed to extract structured information from patient-generated text more reliably by applying preference optimization to correct token-level errors that supervised fine-tuning alone cannot fix [1]. Patient narratives and other unstructured text contain details about lived experiences, social context, and care engagement, but their lack of structure limits use in patient-centered outcomes research [1][2]. The earlier PV-Miner benchmark and PVMinerLLM models were built to extract structured data from such text, yet supervised fine-tuning (SFT) proved insufficient for rare, fine-grained, and unevenly distributed errors, especially in outputs where individual tokens carry critical meaning [1][2]. PVminerLLM2 addresses this by introducing a preference objective with a token-level gated stabilization term, which prevents the degradation of absolute token likelihood during preference optimization [1][2]. The method also uses confusion-aware preference pair construction to capture distinctions that are difficult to separate, and it incorporates token-importance weighting and inverse-frequency reweighing to counter token imbalance and class skew [1][2]. Across multiple model sizes, PVminerLLM2 consistently outperformed strong baselines. The gains reached 4.43% on the Code metric, 3.50% on the Sub-code metric, and 1.55% on the Span metric, and the models surpassed baseline large language models trained with existing preference optimization techniques [1][2]. The supplementary material, code, evaluation scripts, and trained models are publicly available on GitHub [1][2]. The work builds on a broader trend in biomedical language processing where structured extraction from clinical narratives remains a persistent challenge. Prior efforts have shown that even large language models fine-tuned on domain corpora can produce outputs with subtle but clinically significant errors when the target schema is complex and token-sensitive [2]. The PVminerLLM2 approach directly targets this gap by reshaping the optimization signal rather than relying solely on additional annotated data or larger architectures [1][2].