Enhancing Algorithm Learning with Large Language Models: Design and Evaluation of AlgoLLM in Higher Education Practice
DOI:
https://doi.org/10.58557/(ijeh).v5i4.371Keywords:
Algorithm education, Empirical evaluation, Interactive tutoring systems, Large language models, Personalized learningAbstract
Algorithm learning remains challenging in computer science education due to its abstract logic, steep conceptual difficulty, and lack of personalized support in traditional settings. This study presents AlgoLLM, a modular instructional system built on large language models (LLMs) to support students through natural language explanations, code-level guidance, and feedback-based refinement. The system includes four core components: Knowledge Explainer, Exercise Generator, Code Assistant and Debugger, and Feedback Evaluator. A four-week case study was conducted with 60 undergraduate students, comparing a control group using textbooks and an experimental group using AlgoLLM. Paired and independent t-tests showed that the experimental group achieved significantly higher learning gains in post-tests (mean increase of 18.3 percent, Cohen's d = 0.94). Code accuracy and task efficiency also improved. Pearson correlation revealed a moderate relationship between LLM interaction frequency and learning gain. Questionnaire feedback indicated high perceived usefulness, clarity, and satisfaction. These results suggest that LLM-based systems like AlgoLLM can enhance algorithm comprehension and offer scalable, personalized support in technical education
References
Chase, H. (2022, October). LangChain. https://github.com/langchain-ai/langchain
Cormen, T. H., Leiserson, C. E., Rivest, R. L., & Stein, C. (2022). Introduction to Algorithms, fourth edition. MIT Press.
Essel, H. B., Vlachopoulos, D., Tachie-Menson, A., Johnson, E. E., & Baah, P. K. (2022). The impact of a virtual teaching assistant (chatbot) on students’ learning in Ghanaian higher education. International Journal of Educational Technology in Higher Education, 19(1). https://doi.org/10.1186/s41239-022-00362-6
Goslen, A., Kim, Y. J., Rowe, J., & Lester, J. (2025). LLM-Based Student Plan Generation for Adaptive Scaffolding in Game-Based Learning Environments. International Journal of Artificial Intelligence in Education, 35(2), 533–558. https://doi.org/10.1007/s40593-024-00421-1
Holmes, W. (2019). Artificial intelligence in education: Promises and implications for teaching and learning. Center for Curriculum Redesign.
Jundan Wang. (2024). Application and Exploration of Cloud Technology in University Classroom Teaching—Take Cloud Classroom as an Example. International Journal of Interdisciplinary Research and Innovations, 12,(2348–1226), 55–62. https://doi.org/10.5281/ZENODO.10793049
Kamalov, F., Santandreu Calonge, D., & Gurrib, I. (2023). New Era of Artificial Intelligence in Education: Towards a Sustainable Multifaceted Revolution. Sustainability, 15(16), 12451. https://doi.org/10.3390/su151612451
Khan, M., Akbar, M. A., & Kasurinen, J. (2025). Integrating LLMs in Software Engineering Education: Motivators, Demotivators, and a Roadmap Towards a Framework for Finnish Higher Education Institutes (No. arXiv:2503.22238). arXiv. https://doi.org/10.48550/arXiv.2503.22238
Kreijkes, P., Kewenig, V., Kuvalja, M., Lee, M., Vitello, S., Hofman, J., Sellen, A., Rintel, S., Goldstein, D. G., Rothschild, D. M., Tankelevitch, L., & Oates, T. (2025). Effects of LLM Use and Note-Taking On Reading Comprehension and Memory: A Randomised Experiment in Secondary Schools. Elsevier BV. https://doi.org/10.2139/ssrn.5095149
Latif, M., Abbasi, F. L., Ammar, M., Mehmood, B., & Ali, S. (2023). Transforming Personalized Education through AI- Enhanced Ontology Modelling in Dynamic Adaptive Learning Systems. International Journal on Recent and Innovation Trends in Computing and Communication, 12(2).
Michael E. Bernal. (2024). Revolutionizing eLearning Assessments: The Role of GPT in Crafting Dynamic Content and Feedback. Journal of Artificial Intelligence and Technology. https://doi.org/10.37965/jait.2024.0513
OpenAI, Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F. L., Almeida, D., Altenschmidt, J., Altman, S., Anadkat, S., Avila, R., Babuschkin, I., Balaji, S., Balcom, V., Baltescu, P., Bao, H., Bavarian, M., Belgum, J., … Zoph, B. (2024). GPT-4 Technical Report (No. arXiv:2303.08774). arXiv. https://doi.org/10.48550/arXiv.2303.08774
Pinecone. (2025). The vector database to build knowledgeable AI. https://www.pinecone.io/
Sasikala, P., & Ravichandran, R. (2024). Study on the Impact of Artificial Intelligence on Student Learning Outcomes. Journal of Digital Learning and Education, 4, 145–155. https://doi.org/10.52562/jdle.v4i2.1234
Shahzad, T., Mazhar, T., Tariq, M., Ahmad, W., & Ouahada, K. (2025). A comprehensive review of large language models: Issues and solutions in learning environments. Discover Sustainability, 6. https://doi.org/10.1007/s43621-025-00815-8
Tan, K., Yao, J., Pang, T., Fan, C., & Song, Y. (2025). ELF: Educational LLM Framework of Improving and Evaluating AI Generated Content for Classroom Teaching. Journal of Data and Information Quality. https://doi.org/10.1145/3712065
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2025 Shitong Peng, Yingzhao Lin, Shoukang Yu, Jiajun Wu
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
