Trace2Policy: From Expert Behavior Traces to Self-Evolving Decision Agents

A new system called Trace2Policy can recover tacit decision rules from expert behavior and refine them into deterministic code that outperforms large language models on compliance-sensitive audit tasks, according to research posted to arXiv [1]. The core mechanism, named Error-driven Iterative Skill Refinement (EISR), treats a human-readable rule document as its optimization target. Each round executes the rules on a validation set, clusters errors by root cause into MISSING, WRONG, or CONFLICT types, applies targeted patches, and commits only those that pass a regression gate [1][2]. The researchers identify rule quality — not model capability — as the dominant performance lever for this class of skewed-base-rate decision tasks [1][2]. Across five LLMs, one-shot distillation plateaus near 70% accuracy on the deployed pool. Eight EISR rounds lift the same rules to 79.6% when compiled into deterministic Python, requiring zero LLM calls at inference [1][2]. Execution form compounds the gain: the same EISR-refined content runs 9.8 percentage points higher as compiled Python than as an LLM prompt [1][2]. The system was deployed for 22 days at a major logistics carrier, processing 3,349 audit cases. The compiled pipeline outperformed the pure-LLM baseline it replaced, which had achieved 72.7% accuracy. On these calibrated workloads, re-enabling LLM fallback monotonically degraded accuracy [1][2]. An LLM-driven variant called Auto-EISR reproduces the refinement at $5–$10 per cycle, compared with roughly 70 expert-hours for manual iteration. The variant transferred to four public benchmarks spanning legal reasoning and process-mining decisions without re-engineering [1][2]. The paper was submitted to arXiv on June 9, 2026, under the artificial intelligence category [1]. arXiv, an open-access repository of electronic preprints founded in 1991, now receives about 24,000 articles per month and hosts over two million papers [6]. The research appears alongside experimental community tools developed through arXivLabs, a framework that allows collaborators to build features directly on the platform while adhering to values of openness, community, excellence, and user data privacy [5].