The Order Matters: Sequential Fine-Tuning of LLaMA for Coherent Automated Essay Scoring

A new study finds that the order in which a language model learns to assess essay components significantly affects its scoring accuracy, with a sequential curriculum outperforming isolated or randomized training approaches on a standard benchmark corpus. The research, posted to arXiv on June 9, 2026, examines automated essay scoring (AES) systems that must evaluate interdependent discourse elements such as lead, claim, evidence, and conclusion. Most current AES methods treat these elements separately, which the authors argue damages coherence and the model's ability to generalize [1]. The team investigated fine-tuning Meta's LLaMA-3.1-8B model using parameter-efficient Low-Rank Adaptation (LoRA) combined with 4-bit quantization, a technique that reduces memory requirements while preserving performance [2]. The experiments, conducted on the PERSUADE 2.0 corpus, compared three distinct training curricula. The Sequential approach progressively fine-tuned the model on lead, then position, then claim, then evidence, and finally conclusion. The Independent method trained separate task-specific models for each element. The Randomized curriculum shuffled multi-task training data [2]. The Sequential strategy delivered the strongest overall results. It achieved an F1 score of 65% for evidence and 87% for conclusion, with corresponding accuracy figures of 63% and 85% [2]. These scores surpassed those of the independently trained models. Notably, the sequentially fine-tuned 8-billion-parameter model outperformed a general-purpose LLaMA-70B baseline on the conclusion task, despite the latter's substantially larger capacity [2]. Randomized training did show an advantage in one area, improving position scoring to a 57% F1 score, but its performance was less consistent across other discourse elements [2]. The findings suggest that aligning the training curriculum with the natural structure of discourse can materially improve automated assessment. The authors also argue that small, task-optimized models offer a practical path to scalable and cost-effective evaluation, challenging the assumption that larger general-purpose models are always necessary [2]. The researchers have released their templates and implementation details to support reproduction and further work on curriculum design for educational natural language processing [2].