FENCE: A Financial and Multimodal Jailbreak Detection Dataset

Researchers have released FENCE, a bilingual Korean-English multimodal dataset designed to train and evaluate jailbreak detectors for financial applications, addressing a gap in safety resources for vision language models. The dataset, detailed in a paper by Mirae Kim, Seonghun Jeong, and Youngjun Kwak, targets vulnerabilities in Vision Language Models (VLMs), which process both text and images and therefore present broader attack surfaces than text-only systems [1][4]. Jailbreaking—crafting inputs to bypass model safeguards—poses a significant risk to deploying large language models and VLMs, yet detection resources remain scarce, especially in finance [1][2]. FENCE comprises 5,000 finance-domain text-image pairs originally constructed in Korean and translated into English, yielding 10,000 total samples across more than 15 financial categories [4]. The queries are derived from frequently asked consumer questions to ensure domain realism, and images are collected via keyword-based crawling and fused with text using diverse layout strategies [4][5]. Labels were assigned using GPT-4o as an evaluator, with human validation confirming 95 percent agreement [4]. Experiments on 15 commercial and open-source VLMs revealed consistent vulnerabilities. GPT-4o showed measurable attack success rates, while open-source models displayed greater exposure [1][4]. A baseline binary classifier trained on FENCE achieved 99 percent in-distribution accuracy and maintained strong performance on external benchmarks [1][4]. The dataset is intended for training and fine-tuning guardrail models, not solely for evaluation, distinguishing it from prior benchmarks that covered narrower categories [5]. The name FENCE symbolizes a protective boundary against harmful queries, reflecting its goal of reinforcing safety in high-stakes financial AI systems [5]. The paper includes a content warning that example data may be offensive [1][4].