AI-Driven Assessment of Human Tutors: Linking Training Performance to Real-Life Practice

A new AI-driven system can evaluate human tutors by analyzing transcripts of their real-life teaching sessions, linking performance during training directly to classroom outcomes, according to research published on arXiv [1]. The system, detailed in a paper submitted June 17, uses the generative AI model Gemini-2.5-pro to assess both open-ended responses during training and authentic tutoring sessions, measuring how well tutors transfer skills to real-world application [1]. The study involved 86 human tutors who provided remote math instruction and completed six scenario-based lessons, resulting in an average learning gain of 7.4% [1]. Researchers used mixed-effects models across 405 session-to-lesson pairs and found that training performance significantly predicted real-life transcript scores, with an effect size of 0.25 SD [1]. A comparison of models indicated that averaging open-response and multiple-choice performance during training best predicted real-life tutoring quality, though open responses alone were more predictive than multiple-choice scores [1]. Exploratory analysis showed that after training, tutors encountered pedagogical opportunities to apply their skills more frequently, rising from 61.1% to 68.9%, and their execution quality within those opportunities improved from 65.5% to 68.1% [1]. An interrupted time series analysis suggested these gains were part of a gradual trend rather than an immediate result of the training intervention [1]. The authors released open datasets, AI prompts, and scoring rubrics to support transparency and reproducibility [1]. The work appears on arXiv, a preprint platform that has integrated with Hugging Face Spaces to allow researchers to link interactive demos directly alongside papers [6][7]. The broader field of educational technology draws on disciplines including artificial intelligence and computer science to facilitate learning, with AI applications spanning language translation, decision-making, and e-commerce [3][4].