Evaluation-Strategy Gap in Fault Diagnosis of Deep Learning Programs

A new study finds that fault-diagnosis tools for deep learning programs lose significant accuracy when tested on code they have not seen before, exposing a gap in how these systems are evaluated. [1] Researchers constructed a corpus called DynFault, containing 5,542 fault-injected training traces drawn from 38 real-world deep learning programs, to measure the performance drop. [1] They found a gap of 0.190 in balanced accuracy when comparing the standard within-program cross-validation approach against a method that holds out entire programs. [1] The investigation traced the cause to program-level structure embedded in the diagnostic features. [1] The work, posted on the preprint server arXiv, examined two alternative runtime feature sets to see how they behaved on unseen programs. [1] Curvature features proved useful for detecting training instability in new programs, while optimizer and activation features only helped on programs that were part of the training set. [1] arXiv, which hosts the paper, is an open-access repository that has grown to a submission rate of roughly 24,000 articles per month as of late 2024. [6] The findings arrive as the broader field of artificial intelligence safety faces scrutiny over whether protective measures are keeping pace with rapid capability advances. [4] AI safety research encompasses robustness, monitoring, and alignment, and the field gained significant attention in 2023 alongside progress in generative AI. [4] During the 2023 AI Safety Summit, both the United States and the United Kingdom established dedicated AI Safety Institutes. [4] High-quality datasets, such as the DynFault corpus, are considered integral to machine learning research, though producing labeled training data remains expensive and time-consuming. [3] The study’s authors argue that evaluating fault diagnosis techniques solely within a single program can overstate real-world reliability, and they recommend that future assessments account for program-level variation. [1]