Adaptive feedback from an AI tutor in higher mathematics: a markov model of competence acquisition and the role of feedback latency
Keywords:
artificial intelligence in education, AI tutor, large language models, adaptive feedback, feedback latency, knowledge tracing, Markov chain, higher mathematics, effect sizeAbstract
Large language model tutors return feedback within seconds, whereas in a conventional higher mathematics course it arrives a week later. This paper asks how much of the measured advantage of AI tutoring is attributable to latency as such. A three-state Markov model of competence acquisition is developed in which the per-attempt acquisition probability is an explicit function of feedback delay, with a slip-and-guess observation model linking latent competence to observed performance. Over a semester of 45 practice attempts the model predicts a test score of 73.1 % under an AI tutor against 65.0 % under weekly feedback — a gain of 8.1 percentage points, d ≈ 0.54, consistent with meta-analyses of intelligent tutoring systems. The advantage has a half-life of about 41 hours in feedback delay: it is almost fully preserved within an hour, but a third is lost at 24 hours. Verification requires 54 students per group
References
1. Decree of the President of the Republic of Uzbekistan No. PF-5847 of 8 October 2019 “On approval of the Concept for the Development of the Higher Education System until 2030”. – Tashkent: Lex.uz, 2019.
2. Corbett A.T., Anderson J.R. Knowledge tracing: Modeling the acquisition of procedural knowledge // User Modeling and User-Adapted Interaction. – 1995. – Vol. 4, No. 4. – P. 253–278.
3. VanLehn K. The relative effectiveness of human tutoring, intelligent tutoring systems, and other tutoring systems // Educational Psychologist. – 2011. – Vol. 46, No. 4. – P. 197–221.
4. Ma W., Adesope O.O., Nesbit J.C., Liu Q. Intelligent tutoring systems and learning outcomes: A meta-analysis // Journal of Educational Psychology. – 2014. – Vol. 106, No. 4. – P. 901–918.
5. Kulik J.A., Fletcher J.D. Effectiveness of intelligent tutoring systems: A meta-analytic review // Review of Educational Research. – 2016. – Vol. 86, No. 1. – P. 42–78.
6. Kestin G., Miller K., Klales A., Milbourne T., Ponti G. AI tutoring outperforms in-class active learning: an RCT introducing a novel research-based design in an authentic educational setting // Scientific Reports. – 2025. – Vol. 15. – Art. 97652.
7. Kulik J.A., Kulik C.C. Timing of feedback and verbal learning // Review of Educational Research. – 1988. – Vol. 58, No. 1. – P. 79–97.
8. Shute V.J. Focus on formative feedback // Review of Educational Research. – 2008. – Vol. 78, No. 1. – P. 153–189.
9. Hattie J., Timperley H. The power of feedback // Review of Educational Research. – 2007. – Vol. 77, No. 1. – P. 81–112.
10. Cohen J. Statistical Power Analysis for the Behavioral Sciences. 2nd ed. – Hillsdale: Lawrence Erlbaum Associates, 1988. – 567 p.


