From Memorization to Creation: Evaluating the Cognitive Depth of LLM-Generated Educational Questions

A new study evaluates six large language models on their ability to generate educational questions that move beyond simple recall, finding that specific prompting strategies can significantly improve cognitive depth and reduce repetition. The research, submitted for publication on May 6, 2026, analyzes 20,700 questions generated across computer science, K-12 math, and social-science domains [1]. The evaluation used Bloom's Taxonomy, a hierarchical framework for classifying learning objectives, to measure whether the questions stimulated higher-order thinking skills like analysis and creation, rather than just rote memorization [1]. The study comes amid a broader debate in education about the role of generative AI, where initial bans driven by cheating concerns are giving way to explorations of thoughtful integration into assessments [3]. A key finding was the impact of prompt design. A fine-grained prompting strategy applied to the Qwen2.5-7B-Instruct model reduced question repetitiveness by 24.45% [1]. The same approach increased the proportion of higher-order cognitive level outputs by 11.53% for InternLM3-8B-Instruct [1]. The researchers also introduced new quantitative metrics, including one for cognitive shift intensity called CogShift, to measure how effectively models transition between different levels of thinking [2]. InternLM3 demonstrated superior performance in these multi-level transitions [1]. The study's focus on cognitive-aware prompt engineering is relevant as large language models, which are machine learning models trained on vast amounts of text, become more common in personalized learning systems [4]. The development of these models is a global, competitive field. Chinese company DeepSeek, for example, recently launched its R1 model, which it claims was trained for US$6 million, a fraction of the cost of comparable U.S. models [5]. The study's interpretability analysis also found correlations that enhance the transparency of Chain-of-Thought prompting, a technique where models show their reasoning steps [1]. The authors conclude that their work provides benchmarks for deploying LLMs in education in a way that prioritizes cognitive development [1].