When Large Language Models Fail in Healthcare: Evaluating Sensitivity to Prompt Variations

Large language models used in clinical settings remain dangerously sensitive to small changes in how a question is phrased, according to a systematic analysis posted to arXiv on 5 June 2026. The study found that even minor rewording can alter a model's medical advice, and targeted adversarial prompts can produce harmful outputs. The paper evaluated general-purpose models such as GPT-3.5 and Llama3 alongside domain-specific systems including ClinicalBERT, BioLlama3, and BioBERT using the MedMCQA benchmark [1][2]. Researchers categorized perturbations into natural and adversarial types and measured their effect on consistency, accuracy, and reliability in clinical reasoning tasks [2]. "Medical LLMs are not intrinsically safe," the authors concluded, noting that models that change diagnoses because of reworded inputs or hallucinate medications when slightly rephrased cannot be reliably trusted by clinicians [2]. The study documented a pattern of selective fragility. Models tended to withstand simple lexical substitutions or paraphrasing but frequently broke down under syntactic reordering or misleading contextual cues [1][2]. Adversarial manipulations led to clinically dangerous outputs, including recommendations for incorrect dosages and omissions of critical findings [2]. The unpredictability was observed across both general-purpose and domain-specific systems, challenging assumptions that fine-tuning on medical corpora alone confers safety [1][2]. These findings arrive as health systems in several high-income countries face mounting pressure. Canada's publicly funded Medicare system covers roughly 70 percent of healthcare needs, with the remaining 30 percent — including prescription drugs and mental-health services — paid through private insurance or out-of-pocket spending [3]. The Commonwealth Fund's 2021 report ranked Canada second-to-last among 11 high-income countries, citing long wait times, poor after-hours care availability, and a shortage of healthcare professionals and hospital capacity [3]. In such strained environments, the appeal of automated clinical decision support is clear, but the new sensitivity analysis underscores that reliability thresholds for deployment remain unmet [1][2]. The vulnerability to syntactic reordering and misleading cues echoes a broader challenge in artificial intelligence: systems lack the contextual understanding that humans develop through social cognition. Research on theory of mind — the capacity to ascribe mental states to others — shows that this faculty depends on regions including the medial prefrontal cortex and the temporoparietal junction, and that deficits in it are associated with conditions such as autism spectrum disorder and schizophrenia [4]. The double empathy problem, first articulated in 2012 by autism researcher Damian Milton, further posits that communication breakdowns often arise from bidirectional mismatches in disposition and experience rather than from a unilateral deficit [5]. A 2025 systematic review of 52 papers found that autistic people generally report positive interpersonal relations when interacting with other autistic people, challenging the notion that social skills are universally impaired across contexts [5]. The LLM sensitivity documented in the arXiv paper suggests a comparable brittleness: the models do not fail uniformly but collapse under specific syntactic and contextual shifts that a human clinician would navigate without difficulty [1][2].