Trust Between AI Agents: Measuring Formation, Breakage, and Recovery, with Implications for Governing Multi-Agent Systems

22d ago · Global · primary source: export.arxiv.org

A new study proposes a behavioral measure to quantify trust between AI agents, finding that several frontier models substantially reduce costly verification of teammates they deem reliable, with implications for governing multi-agent systems [1]. The research, submitted in 2026, introduces a framework based on a cooperative survival game where checking a teammate's work consumes resources but trusting an incorrect answer can be fatal [1]. Relative to a memoryless version of the same model, reduced verification provides an observable measure of trust [2]. When paired with a consistently reliable teammate, four of six frontier model snapshots — Claude Opus 4.6, Claude Sonnet 4.6, GPT-5.1, and Gemini 3.1 Pro — reduced verification by roughly 60-85%, while two smaller snapshots showed little or no such adjustment [1][2]. Failures reversed this discount, but models diverged in their responses. Some concentrated renewed scrutiny on the teammate that erred, while others became more cautious toward the entire team [1]. Recovery from broken trust was slower than its initial formation, and clustered failures sustained suspicion far longer than the same number of failures spread apart [2]. The study found that models which formed trust verified less, decided more quickly, and achieved higher payoffs, while persistent over-verification was associated with indecision rather than safety [1]. The findings arrive as multi-agent AI systems draw increasing research attention. A separate 2026 study evaluating 12 multi-agent LLM collaboration topologies for software architecture design found that structural adversarial variants and cross-model review configurations ranked highest across 520 experimental runs, while parallel-merge approaches consistently underperformed due to token starvation and incoherent outputs [3]. That work used Claude Opus 4.6 and Claude Sonnet 4.6 among its three independent automated evaluators [3]. Claude Opus 4.6 and Claude Sonnet 4.6 are developed by Anthropic, a San Francisco-based AI company founded in 2021 by former OpenAI members with a focus on AI safety [7]. Anthropic's Claude series, first released as a chatbot in March 2023, is trained using "constitutional AI" techniques aimed at improving ethical and legal compliance [5]. Since the Claude 3 generation, models have typically been released in three sizes: Haiku, Sonnet, and Opus [5]. The company was privately valued at an estimated $965 billion in May 2026 [7]. A separate benchmark study of MRI physics knowledge found that Claude Opus 4.6 and Claude Sonnet 4.6 achieved overall multiple-choice accuracy between 93.2% and 97.1%, but accuracy fell sharply when answer options were removed — dropping to 58.4% to 61.1% for frontier models in stem-only conditions, and as low as 13.8% to 29.8% for vendor-specific scanner operations questions [4]. The researchers concluded that high multiple-choice performance can mask weak free-text recall [4]. The trust-measurement study's authors argue that trust dispositions can be measured before deployment and that calibration, rather than maximal suspicion, should be the central concern in governing multi-agent AI systems [1][2].

applicationmodel-releaseresearch-papertool-release

Background sources we checked (6)
  • arxiv.org ↗ As language-model agents increasingly work in teams, each agent must decide how much to trust its teammates. Yet we lack a standard way to measure trust between AI agents. We propose a behavioral measure based on costly verification. In a cooperative survival game, checking a tea…
  • arxiv.org ↗ We present a controlled experiment evaluating 12 multi-agent LLM collaboration topologies for software architecture design. Using a $2\times2\times2$ factorial design (Authority $\times$ Roles $\times$ Dynamics), we conducted 520 experimental runs across 8 design tasks of varying…
  • arxiv.org ↗ Background: Existing MRI LLM benchmarks rely mainly on review-book multiple-choice questions, where top proprietary models already score highly, limiting discrimination. No systematic benchmark has evaluated vendor-specific scanner operational knowledge central to research MRI pr…
  • en.wikipedia.org ↗ Claude is a series of large language models developed by American software company Anthropic. Claude was released as an AI-based chatbot in March 2023. It is also used in AI-assisted software development. Claude is trained using "constitutional AI", a technique developed by Anthr…
  • en.wikipedia.org ↗ A large language model (LLM) is a type of machine learning model designed for natural language processing tasks such as language generation. LLMs are language models with many parameters, and are trained with self-supervised learning on a vast amount of text.…
  • en.wikipedia.org ↗ Anthropic PBC is an American artificial intelligence (AI) company headquartered in San Francisco, California. It has developed a series of large language models (LLMs) named Claude and has a focus on AI safety. Anthropic was founded in 2021 by former members of OpenAI, including …

Sources

Spot something wrong? Report an issue