Cross-Dataset Bloom Question Classification: Supervised Models and Prompted LLMs

Supervised machine learning and deep learning models degrade substantially when classifying educational questions on unseen datasets, while large language models show greater stability, according to a new study that evaluated cross-dataset generalization across five question banks [1]. The work, submitted to arXiv on 22 April 2026 by Mohammadreza Molavi, addresses a persistent gap in automated Bloom’s taxonomy classification: prior ML and DL approaches reported strong within-dataset results but were rarely tested in cross-dataset settings, leaving real-world generalizability unclear [1][2]. The study evaluated existing supervised methods alongside LLMs using multiple prompting strategies [1]. Supervised models dropped sharply when applied to datasets different from those they were trained on [1][3]. In contrast, LLM performance showed substantially less sensitivity to dataset shifts, indicating stronger cross-dataset robustness [3][4]. The best prompting strategy combined in-context examples with course-specific action verbs, reaching weighted F1-scores of up to 0.84 on one dataset [3][4]. Large language models are neural networks trained on vast text corpora for natural language processing tasks, typically built on transformer architectures that are more parallelizable than earlier recurrent models [6]. The study’s LLM findings align with separate zero-shot evaluations in which models such as OpenAI GPT-4o-mini and Google Gemini-1.5-Pro reached accuracy scores of 0.72 to 0.73 without any task-specific training [5]. Those experiments also found that LLaMA and Claude were more prompt-sensitive and less accurate, suggesting differences in architectural and training approaches across LLM families [5]. The arXiv paper notes that LLMs do not surpass the within-dataset test performance of fine-tuned models, but their comparatively stable performance across diverse datasets highlights their potential as a practical alternative in heterogeneous educational settings [3][4]. This robustness is particularly valuable given the subjective and context-dependent nature of Bloom’s taxonomy, where labeled data from a single instructional context may not generalize reliably to unseen sources [2][4]. Based on the best-performing prompting strategy, the researchers also presented a lightweight user interface that supports instructors in automatically classifying large question banks; a usability study indicated low workload and high usability [1][2]. The first version of the paper was submitted at 13:40:25 UTC and weighs 245 KB [1].