EiCAP: Beyond Fluency, Probing and Improving Emotional Intelligence in LLMs via Psychologically Grounded Multi-Turn Dialogue

A new framework called EiCAP aims to measure and improve emotional intelligence in large language models, addressing a gap as these systems move into mental health support, education, and crisis response roles [1][2]. The work, led by Ehsaneddin Asgari and colleagues, introduces a six-layer taxonomy of emotional intelligence grounded in psychology [1][2]. This taxonomy is operationalized into two resources: EiCAP-Bench, a multi-turn evaluation suite with 3,174 probes across 24 subcategories, and EiCAP-SFT, a supervision corpus of 152,820 dialogues aligned to the same taxonomy [1][2]. The benchmark uses a one-vs-three forced-choice format with cross-turn dependencies designed to reflect real conversational demands [2]. A central finding is that generic conversational fine-tuning does not confer emotional intelligence. When the researchers fine-tuned a model on UltraChat, a general dialogue dataset, the macro score on EiCAP-Bench was 24.6%, statistically indistinguishable from the 25% chance level [1][2]. No significant gain was observed in any of the 24 subcategories [2]. In contrast, applying an EI-grounded Low-Rank Adaptation, or LoRA, directly to the Qwen-2.5-7B-Base model produced substantial improvements. Using approximately 0.8% of the model's parameters, the fine-tuned system reached a macro score of 75.33% [1][2]. This represents a gain of 51.7 percentage points over the base model and 37.1 percentage points over the instruct-tuned version [1][2]. An ablation study revealed that a pre-training stage on UltraChat was counterproductive, reducing performance by 21.4 percentage points [1][2]. The authors conclude that direct EI-grounded training is both necessary and sufficient for imparting emotional intelligence to these models [2]. The research arrives as large language models are increasingly deployed in contexts where emotional sensitivity is critical. The United Nations has noted that rising inequalities and mental health challenges have been exacerbated by global crises, including the COVID-19 pandemic, which set back development goals related to health and well-being [6]. The EiCAP framework offers a structured method for evaluating whether AI systems can meet the conversational demands of such sensitive applications [1][2].