AutoSpec: Safety Rule Evolution for LLM Agents via Inductive Logic Programming

A new framework called AutoSpec automatically refines expert-written safety rules for large language model agents, using inductive logic programming to cut false alarms while catching genuine threats, according to research posted on arXiv [1]. Large language model agents that can use tools and act in environments carry inherent risks, including executing destructive commands or leaking sensitive data [1]. Current safety methods split between hand-crafted rules, which are interpretable but brittle, and neural classifiers, which lack the transparency required for safety-critical deployments [1]. The AutoSpec framework, described in a paper submitted to arXiv on 23 June 2026, aims to resolve that tension by evolving rules through counterexample-guided inductive synthesis, or CEGIS, steered by inductive logic programming [1]. AutoSpec starts from a set of expert-designed rules and a stream of execution traces annotated as safe or unsafe [1]. It evaluates the rules, mines false-positive and false-negative counterexamples, and uses inductive logic programming to identify predicates that discriminate between them [1]. The system then generates candidate rule edits and verifies them, selecting the revision that best balances precision and recall [1]. The authors report that the inductive logic programming guidance prunes the exponential search space of possible edits by spotting predicates that appear frequently in one class of error but rarely in the other [1]. The framework was tested on 291 execution traces across code execution and embodied agent domains [1]. AutoSpec lifted the F1 score of the safety rules to 0.98 in the code execution domain and 0.93 in the embodied agent domain [1]. It achieved up to a 94 percent reduction in false positives while maintaining high recall, and converged within four to five iterations [1]. The inductive logic programming-guided approach delivered up to 4.8 times higher F1 than a heuristic CEGIS baseline [1]. The resulting rules are human-readable, auditable, and generalized to scenarios not seen during training [1]. The paper appears on arXiv, the open-access repository that hosts preprints across physics, computer science, and other fields [6]. As of late 2024, arXiv was receiving roughly 24,000 new articles per month and had surpassed two million total submissions by the end of 2021 [6]. The repository’s arXivLabs program, which enables community collaborators to build tools on top of the platform, operates under guidelines that require partners to uphold openness, community, excellence, and user data privacy [4]. The AutoSpec abstract page displays several such Labs integrations, including Bibliographic Explorer and Connected Papers, which help readers navigate citation networks and discover related work [5].