Phishing Email Detection Using Large Language Models

A new framework called LLMPEA uses large language models to detect phishing emails with accuracy above 90 percent, according to research posted on arXiv. The study also warns that the same models remain vulnerable to adversarial attacks that exploit their underlying architectures. The paper, authored by Shaohu Zhang and submitted in December 2025, proposes LLMPEA as a detection system designed to counter multiple attack vectors simultaneously, including prompt injection, text refinement, and multilingual phishing campaigns [1][2]. The researchers evaluated three frontier models — GPT-4o, Claude Sonnet 4, and Grok-3 — and found that while detection rates exceeded 90 percent, the systems could still be exploited by coordinated multi-vector attacks [2]. “Current LLMs require substantial hardening before deployment in email security systems,” the authors write, noting that real-world attackers increasingly combine several vulnerabilities in a single campaign [2]. Phishing remains the most prevalent type of cybercrime globally, and its sophistication has grown sharply with the integration of generative AI, which enables automated, hyper-targeted campaigns at scale [4]. The FBI’s Internet Crime Complaint Center has historically ranked phishing at the top of reported cyber intrusions, and phishing attacks against businesses rose from 72 percent in 2017 to 94 percent in 2023 [4]. The LLMPEA study arrives as security researchers grapple with a fundamental tension: the same large language models that can filter malicious emails can also be weaponized to generate them [6]. A separate study, also posted on arXiv, introduced the Trustworthiness Calibration Framework, a methodology for evaluating phishing detectors beyond raw accuracy [3]. That work assessed models across three dimensions — calibration, consistency, and robustness — and found that reliability varies independently of benchmark scores [3]. GPT-4 achieved the strongest overall trust profile among the tested models, followed by LLaMA-3-8B and DeBERTa-v3-base [3]. The finding reinforces the LLMPEA authors’ caution that high detection accuracy alone does not guarantee safe deployment [2][3]. No single anti-spam or anti-phishing technique eliminates the problem entirely; every method involves trade-offs between false positives and false negatives [5]. The LLMPEA framework adds to a growing body of research that seeks to harden LLM-based defenses before they are integrated into production email systems [2][5].