DoubtProbe: Black-Box Jailbreak Defense via Structural Verification and Semantic Auditing

A new inference-time defense framework called DoubtProbe combines structural verification with semantic auditing to reduce black-box jailbreak success rates on large language models, according to research posted to arXiv on June 15, 2026 [1]. The framework targets a practical problem: as LLMs are deployed in user-facing systems, attackers increasingly evade safety alignment by reorganizing harmful instructions rather than removing them, making prompt-level defenses unstable [1]. DoubtProbe approaches the issue as a consistency-checking task. Its structural branch extracts a structured representation from the original request, reconstructs the request under representation constraints, and flags information-preservation failures. A parallel semantic branch audits the original prompt directly [1]. On the Qwen2.5-72B model, DoubtProbe reduced the JBB attack success rate from 0.293 to 0.100 and the CodeAttack success rate from 0.152 to 0.001 [1]. False positive rates on benign-request benchmarks remained low: 0.022 on AlpacaEval and 0.016 on OR-Bench [1]. The researchers also tested backbone transfer to Llama-3.1-70B and reported that the defense-utility trade-off remained stable [1]. Existing black-box defenses often depend on known-attack coverage, prompt-level semantic judgment, or local runtime control, each of which can degrade under evolving prompt packaging and expression rewriting [1]. The authors argue that structural inconsistency signals offer a more generalizable basis for jailbreak defense, particularly when paired with semantic auditing [1]. The work appears as LLM safety research continues to grapple with adversarial prompt engineering. While the DoubtProbe paper does not release a public implementation with its preprint, the abstract and results are available on arXiv [1]. The framework’s dual-branch design reflects a broader trend in machine learning security toward layered verification, analogous to how transfer learning has been used to improve model robustness across related tasks in other domains, such as catalyst informatics, where models trained on one dataset are fine-tuned on smaller, specialized datasets to boost performance [4].