Poster: Exploring the Limits of Audio-Based Detection of Turkish Phone Call Scams

A new study examines whether large language models can detect Turkish-language phone scams, a problem researchers say has been neglected outside English-speaking contexts [1]. The work introduces a public dataset of 100 aligned audio-transcript pairs and finds that text-based analysis outperforms direct audio processing [1]. Scam phone calls disproportionately harm vulnerable communities, yet detection research has concentrated on English and other high-resource languages [1]. In low-resource settings such as Turkish, the challenge is compounded by a lack of annotated data and limited technological defenses [1]. The study, posted to arXiv on 23 June 2026, investigates how LLMs can fill this gap [1]. The authors assembled the first public multi-modal dataset for Turkish scam detection, comprising 100 aligned audio-transcript pairs of scam and benign conversations [1]. They evaluated seven LLMs from three model families — Gemini 2.5 (Flash, Flash-Lite, Pro), GPT-4o, and Qwen (Max, Plus, Turbo) — under three input conditions: raw audio, automatic speech-to-text transcripts, and transcripts refined by a native speaker [1]. Transcript-based inputs consistently outperformed direct audio processing across the tested models [1]. The study also found that human-corrected and uncorrected transcripts performed comparably, suggesting that automatic transcription may be sufficient for practical deployment [1]. The paper frames the work as a call for culturally and linguistically inclusive AI safety research, noting that robust multi-modal systems for fraud prevention remain underdeveloped for many of the world’s languages [1]. The findings arrive as regulators and technology firms globally face pressure to address AI-driven fraud, though most countermeasures have been built and tested on English-language data [2]. The Turkish-language focus highlights a persistent gap in safety evaluations for low-resource languages, where both commercial and open-source models often lack rigorous benchmarking [2].