A Systematic Evaluation of Large Language Models for PTSD Severity Estimation: The Role of Contextual Knowledge and Modeling Strategies

A systematic evaluation of 11 large language models found that providing detailed construct definitions and narrative context significantly improves their ability to estimate PTSD severity, according to a study using clinical data from 1,437 individuals [1]. The research, led by Panagiotis Kaliosis and posted on arXiv, tested state-of-the-art models on their capacity to assess post-traumatic stress disorder severity from natural language narratives paired with self-reported scores [1]. The team varied the contextual knowledge supplied to the models, including subscale definitions, distribution summaries, and interview questions, alongside modeling strategies such as zero-shot versus few-shot prompting and the amount of reasoning effort [1]. When given detailed construct definitions and the context of the narrative, the LLMs’ accuracy exceeded the agreement observed between human raters and self-reported scores [1]. Increased reasoning effort consistently led to better estimation accuracy, the authors reported [1]. The study also found that the performance of open-weight models, such as Llama and DeepSeek, plateaus beyond 70 billion parameters, while closed-weight alternatives like gpt-o3-mini and gpt-5 improve with newer generations [1]. The best overall performance was achieved by ensembling a supervised model with the zero-shot LLMs [1]. Beyond simple agreement with self-reports, the models’ estimates discriminated PTSD severity from depression, anxiety, and alcohol use, and prospectively predicted future mental healthcare expenditure [1]. The findings suggest that contextual knowledge and modeling strategies meaningfully affect both the accuracy and clinical utility of LLM-based mental health assessments [1]. The work builds on a growing body of research applying machine learning to mental health. While the study focused on PTSD, other psychiatric conditions such as anorexia nervosa carry the highest mortality rate of any psychiatric diagnosis, with medical complications that can include osteoporosis, infertility, and heart damage [3]. The ability of computational tools to parse patient narratives could eventually support clinicians who manage complex, high-risk conditions where early and accurate severity estimation is critical. The study’s first submission, dated February 5, 2026, was 245 KB, and a revised version followed on June 14, 2026, at 336 KB [1]. The research was conducted under the auspices of arXivLabs, a framework that allows collaborators to develop and share new features on the arXiv platform while adhering to values of openness, community, and user data privacy [1].