LLMs Prompted for Legal Context Object More: Overrefusal from Small On-Premises LLMs in Criminal Legal Context

Small language models deployed on local devices for legal work systematically refuse to assist on certain topics when given authority-style prompts, according to a new study, raising concerns that selective non-responses could introduce processing-speed biases in criminal cases. The paper, posted to arXiv on 23 June 2026 by Anastasiia Kucherenko, examines several modern small LLMs likely to be used as on-device assistants by legal professionals for tasks such as translation and reformulation [1]. The researchers tested how often the models declined to answer legal prompts under different contextual framings [1]. Without any special prefix, the models provided a baseline refusal rate. When the same prompts were preceded by an authority-style instruction — for example, “you are acting as an assistant of the national supreme court” or “defense lawyer” — refusal rates jumped by a factor of 2 to 20 [1][2]. The study notes that even seemingly innocuous uses of LLMs can introduce bias through case-processing speed if the assistant selectively withholds help on certain topics [2]. A known role-play jailbreak prefix produced mixed results: it sharply increased refusals in some models while barely shifting them in others [1][2]. The instability suggests that small, on-premises LLMs are sensitive to the very contextual framings a real institutional user might naturally introduce [2]. The findings arrive as the legal profession continues to debate the ethical boundaries of AI use. While the validity of LLMs in legal settings remains contested, practitioners are already experimenting with personal models [1][2]. The paper argues that further investigation is essential to minimize opportunities for bias, particularly in criminal legal contexts where uneven assistance could affect outcomes [1][2]. The submission file was 175 KB [1]. The research bundle includes references to tools such as CatalyzeX and DagsHub, which are used for code discovery and data versioning in machine-learning projects [3][5].