When Errors Become Narratives: A Longitudinal Taxonomy of Silent Failures in a Production LLM Agent Runtime

A longitudinal study of a production large-language-model agent runtime found that roughly 70% of silent failures were caught only by human observation, not by automated tests or audits, according to research posted to arXiv on June 12, 2026 [1][2]. The study examined a personal-assistant agent runtime that has been in continuous production since March 2026, running approximately 40 scheduled jobs across eight LLM providers, with a tool-governance proxy and a knowledge-base memory plane [1][2]. The system was defended by 4,286 unit tests and 827 governance checks [1][2]. Over an eight-week period, researchers documented 22 incidents with full root-cause postmortems, identifying one meta-pattern — a failure whose error signal never reaches a human in actionable form — that manifested at least 28 times [1][2]. The authors derived a five-class, mechanism-oriented taxonomy of silent failures: (A) environment and platform quirks, (B) design-assumption mismatches, (C) error swallowing and dilution, (D) chained hallucination and fabrication, and (E) operational omission and forensic blind spots [1][2]. Class D is unique to LLM systems and was described as the most dangerous: the system does not merely fail to report an error, but the LLM transforms it into fluent, plausible narrative delivered to the user [1][2]. The researchers termed this “fail-plausible,” noting that the observer is not just blind to the failure but is “convincingly lied to by the failure itself” [2]. A retrospective audit of 15 incidents found that 0% of silent failures were prevented ex-ante, while 87% were blocked by regression tests after the fact — leading the authors to conclude that audits function as regression engines, not prediction engines [1][2]. Incident latency ranged from 13 hours to 60 days and tracked failure mechanism rather than code complexity; the longest-lived failures resided in the seams between components where no test runs [1][2]. The paper appears on arXiv under the Software Engineering category (cs.SE) and is linked to the arXivLabs framework, which allows collaborators to develop and share new features on the arXiv platform [1]. The research artifacts and all postmortems have been made public [1][2].