Poker Arena: Multi-Axis Profiling of Strategic Reasoning and Memory in LLMs

A new evaluation platform called Poker Arena uses no-limit Texas Hold’em tournaments to assess strategic reasoning and memory in large language models, revealing that scalar leaderboards systematically misrank model capability, according to a paper posted to arXiv on June 11, 2026 [1]. The platform couples a three-layer memory architecture — within-hand context, a model-controlled session notebook, and a cross-session lifetime store — with a nine-axis cognitive profile that decomposes strategic reasoning into interpretable dimensions [1][3]. Those axes span bet sizing, bluffing, opponent reading, composure, adaptability, prediction, strategic mixing, factual accuracy, and positional awareness [3][4]. Five axes rely on deterministic action-log statistics, two use regex matches against ground truth in reasoning text, and two are hybrids that pair deterministic counts with an LLM judge; each axis returns a score between 0 and 1, and the unweighted mean across all nine is reported [3][5]. Seven frontier models were evaluated across 50 sessions of 1,000 hands and a controlled memory ablation [1][2]. Claude Opus 4.6 won the tournament, accumulating +$15,730 in chips and securing 14 first-place finishes [1][2]. Yet the same model ranked only fifth of seven on mean axis score [1][3]. The divergence between chip-based and axis-based rankings underscores the paper’s central finding: multi-axis evaluation surfaces capability structure that scalar leaderboards systematically misrank [1][2]. The memory ablation results further complicate the picture. Persistent memory improved performance for some models while degrading it for others, indicating that model-specific memory interfaces matter more than a single aggregate rank [2][5]. The authors argue that cross-dimensional consistency outweighs peak performance on any single axis, a conclusion drawn from the full 1,000-hand tournament dataset [1][3]. The work was posted on arXiv under the title “Poker Arena: Multi-Axis Profiling of Strategic Reasoning and Memory in LLMs” and is hosted on arXivLabs, a framework for community collaborators developing new features on the arXiv site [1]. The paper contends that prevailing game-play benchmarks collapse heterogeneous reasoning dimensions into a single scalar, leaving the capability structure of frontier LLMs unexamined — a gap Poker Arena is designed to fill [1][2].