Advancing the State-of-the-Art in Empirical Privacy Auditing

Researchers have proposed two new methods to improve privacy auditing in machine learning, one for generating synthetic 'canaries' to test data leakage and another for more efficient one-run privacy auditing in differentially private machine learning.

The first method, described in a paper submitted on June 9, 2026^[1], involves generating synthetic canaries via high-temperature sampling from large language models. These canaries act as high-influence outliers, ensuring high identifiability and strong audits. The researchers also propose auditing synthetic data generated from fine-tuning an auxiliary model on the synthetic data. A second paper, submitted on June 10, 2026, and revised on June 12, 2026^[2], introduces a new method for one-run privacy auditing in differentially private machine learning. This approach is based on the distributional perspective of canary-aligned signals in white-box DP-SGD and leads to tighter privacy lower bounds from a single training run. The new method preserves useful information and is more efficient than previous approaches, which threshold training examples or 'canaries' into binary membership guesses.