When Errors Become Narratives: A Longitudinal Taxonomy of Silent Failures in a Production LLM Agent Runtime
- lab Hugging Face
- lab arXiv
- product CatalyzeX
- product DagsHub
- product Gotit.pub
- product LLM
- product ScienceCast
- product alphaXiv
A study of a production LLM agent runtime identified a five-class taxonomy of errors, with 70% of silent failures detected by human observation, not tests or audits[1].
The study, conducted on a personal-assistant agent runtime in continuous production since March 2026, analyzed 22 incidents with full root-cause postmortems over eight weeks. The runtime had 40 scheduled jobs, 8 LLM providers, a tool-governance proxy, and a knowledge-base memory plane. The taxonomy includes environment and platform quirks, design-assumption mismatches, error swallowing and dilution, chained hallucination and fabrication, and operational omission and forensic blind spots. Chained hallucination and fabrication, unique to LLM systems, is considered the most dangerous type of error. Meanwhile, a separate study on large language model (LLM) agent communication protocols developed a technical taxonomy to classify and analyze these protocols, comprising five dimensions: counterparty, payload, interaction state, discovery mechanism, and schema flexibility[2]. The findings highlight the importance of robust communication protocols in distributed agent networks, with all sampled agent-to-agent protocols combining hybrid payloads with session-state persistence. Most protocols support multiple predefined schemas, and two negotiate schemas at runtime.
applicationresearch-papertool-release
Background sources we checked (8)
- arxiv.org ↗ LLM agent systems increasingly run as long-lived autonomous runtimes: scheduling jobs, calling tools, maintaining memory, and pushing results to humans. We present a longitudinal study of silent failures in one such system: a personal-assistant agent runtime in continuous product…
- arxiv.org ↗ We review thirteen generative systems and five supporting datasets for quantum circuit and quantum code generation, identified through a structured scoping review of Hugging Face, arXiv, and provenance tracing (January-February 2026). We organize the field along two axes: artifac…
- huggingface.co ↗ # Paper Pages Paper pages allow people to find artifacts related to a paper such as models, datasets and apps/demos (Spaces). Paper pages also enable the community to discuss about the paper. ## Linking a Paper to a model, dataset or Space If the repository card (`README.md`) …
- huggingface.co ↗ # How to Add a Space to ArXiv ... Demos on Hugging Face Spaces allow a wide audience to try out state-of-the-art machine learning research without writing any code. Hugging Face and ArXiv have collaborated to embed these demos directly along side papers on ArXiv! ... Thanks to th…
- huggingface.co ↗ Daily Papers - Hugging Face new Get trending papers in your email inbox once a day! Get trending papers in your email inbox! Subscribe # Daily Papers ## byAK and the research community - Daily - Weekly - Monthly Trending Papers https://huggingface.co/papers/date/2026-06-…
- en.wikipedia.org ↗ Hangzhou DeepSeek Artificial Intelligence Basic Technology Research Co., Ltd., doing business as DeepSeek, is a Chinese artificial intelligence (AI) company that develops large language models (LLMs). Based in Hangzhou, Zhejiang, DeepSeek is owned and funded by High-Flyer, a Chin…
- en.wikipedia.org ↗ A large language model (LLM) is a type of machine learning model designed for natural language processing tasks such as language generation. LLMs are language models with many parameters, and are trained with self-supervised learning on a vast amount of text.…
- en.wikipedia.org ↗ Qwen (also known as Tongyi Qianwen, Chinese: 通义千问; pinyin: Tōngyì Qiānwèn) is a family of large language models developed by Alibaba Cloud. Many Qwen models are distributed under the free and open-source Apache 2.0 license, the source-available Qwen License, or the non-commercial…