RAS: Measuring LLM Safety Through Refusal Alignment

Researchers have introduced a white-box method called RAS that evaluates the safety of large language models by examining their internal representations rather than their generated text, according to a paper posted on arXiv [1]. The procedure, named SafeVec, extracts layer-wise refusal directions from a safety-aligned reference model and then measures whether a target model’s hidden states align with those directions when exposed to unsafe or jailbreak prompts [1]. The resulting metric, the Refusal Alignment Score, maps representation-level refusal alignment to a calibrated 0–100 safety score [1]. Across the Llama, Gemma, and Qwen model families, RAS separated aligned models from uncensored and abliterated variants, tracked output-level attack success rate, and proved substantially faster than judge-based evaluation [1]. The authors argue that refusal alignment offers a compact and efficient signal for white-box safety evaluation, sidestepping the expense and sensitivity to judge choice that characterize output-level methods [1]. Large language models are machine learning systems with many parameters, trained on vast amounts of text through self-supervised learning [9]. Their rapid proliferation has intensified the search for reliable safety benchmarks. The paper was submitted to the Cryptography and Security section of arXiv on 24 June 2026 [1]. arXiv, founded in 1991, is an open-access repository that hosts preprints across physics, computer science, and other fields and now receives about 24,000 submissions per month [7]. The SafeVec procedure is designed for settings where evaluators have access to a model’s internal weights, a condition that distinguishes it from black-box audits that rely solely on prompted outputs [1]. By selecting stable layer windows where safe and unsafe behaviors are separable, the method aims to produce scores that remain consistent even when the underlying question bank changes [1]. The paper appears on arXiv’s abstract page alongside experimental community tools developed under the arXivLabs framework, which allows third-party collaborators to build features such as citation explorers and code finders while adhering to arXiv’s values of openness and user-data privacy [5][6].