What counts as evidence of learning when students use generative AI? A critical integrative review of assessment validity in higher education

Authors

  • Wenjie Peng College of Physical Education, Southwest University, China

Keywords:

Academic Integrity, Assessment Validity, Generative Artificial Intelligence, Higher Education, Process Evidence

Abstract

Generative artificial intelligence (GenAI) can support feedback, practice, and drafting, but it weakens a familiar assumption of take-home assessment: that the submitted product reliably records students' own learning. Higher education now offers many AI-aware assessment proposals, yet their recommendations often overlap, and the evidential purpose of each safeguard remains unclear. This critical integrative review examined 31 verified works published from 2021 to August 2026, comprising 26 primary empirical, technical, implementation, or document-analysis studies and five review or conceptual papers. Sources were coded for the learning claim at stake, the role allowed for GenAI, the evidence collected, the inference supported, the implementation burden, and the equity risk. The review found that detector scores cannot establish learning, authentic tasks alone remain vulnerable, and no single process record proves independent competence. Stronger designs combine evidence channels: a final product, selected process traces, a short oral or live check, and, where retention matters, delayed unaided performance. Building on this finding, the article presents an evidence-calibration model that asks instructors to define the learning claim, locate the AI-sensitive link in the inference, choose the smallest useful evidence bundle, and match verification intensity to stakes, workload, access, and student rights. The model is a design aid rather than a validated instrument. It offers a practical basis for assessment decisions while clarifying where further empirical testing is needed. The review contributes by synthesizing overlapping recommendations, making each safeguard's evidential purpose explicit, and foregrounding equity and student rights in AI-aware assessment design

References

Abbas, M., Jam, F. A., & Khan, T. I. (2024). Is it harmful or helpful? Examining the causes and consequences of generative AI usage among university students. International Journal of Educational Technology in Higher Education, 21(1), Article 10. https://doi.org/10.1186/s41239-024-00444-7

Ahangama, N. (2026). Designing assessments in the generative AI era: A tailored assessment framework for ICT tertiary education. International Journal of Educational Technology in Higher Education, 23, Article 9. https://doi.org/10.1186/s41239-026-00582-0

Bastani, H., Bastani, O., Sungu, A., Ge, H., Kabakcı, Ö., & Mariman, R. (2025). Generative AI without guardrails can harm learning: Evidence from high school mathematics. Proceedings of the National Academy of Sciences, 122(26), e2422633122. https://doi.org/10.1073/pnas.2422633122

Bissessar, C. (2026). To use or not to use ChatGPT and assistive artificial intelligence tools in higher education institutions? The modern-day conundrum - students' and faculty's perspectives. Equity in Education & Society, 5(2), 132-144. https://doi.org/10.1177/27526461231215083

Carolus, A., Koch, M. J., Straka, S., Latoschik, M. E., & Wienrich, C. (2023). MAILS - Meta AI literacy scale: Development and testing of an AI literacy questionnaire based on well-founded competency models and psychological change- and meta-competencies. Computers in Human Behavior: Artificial Humans, 1(2), 100014. https://doi.org/10.1016/j.chbah.2023.100014

Chan, C. K. Y., & Hu, W. (2023). Students' voices on generative AI: Perceptions, benefits, and challenges in higher education. International Journal of Educational Technology in Higher Education, 20(1), Article 43. https://doi.org/10.1186/s41239-023-00411-8

Furze, L., Perkins, M., Roe, J., & MacVaugh, J. (2024). The AI Assessment Scale (AIAS) in action: A pilot implementation of GenAI-supported assessment. Australasian Journal of Educational Technology, 40(4), 38-55. https://doi.org/10.14742/ajet.9434

Hornberger, M., Bewersdorff, A., & Nerdel, C. (2023). What do university students know about artificial intelligence? Development and validation of an AI literacy test. Computers and Education: Artificial Intelligence, 5, 100165. https://doi.org/10.1016/j.caeai.2023.100165

Johnston, H., Wells, R. F., Shanks, E. M., Boey, T., & Parsons, B. N. (2024). Student perspectives on the use of generative artificial intelligence technologies in higher education. International Journal for Educational Integrity, 20(1), Article 2. https://doi.org/10.1007/s40979-024-00149-4

Kestin, G., Miller, K., Klales, A., Milbourne, T., & Ponti, G. (2025). AI tutoring outperforms in-class active learning: An RCT introducing a novel research-based design in an authentic educational setting. Scientific Reports, 15(1), Article 17458. https://doi.org/10.1038/s41598-025-97652-6

Kofinas, A. K., Tsay, C.-H., & Pike, D. (2025). The impact of generative AI on academic integrity of authentic assessments within a higher education context. British Journal of Educational Technology, 56(6), 2522-2549. https://doi.org/10.1111/bjet.13585

Kong, S.-C., Cheung, W. M.-Y., & Zhang, G. (2021). Evaluation of an artificial intelligence literacy course for university students with diverse study backgrounds. Computers and Education: Artificial Intelligence, 2, 100026. https://doi.org/10.1016/j.caeai.2021.100026

Kong, S.-C., Cheung, W. M.-Y., & Zhang, G. (2022). Evaluating artificial intelligence literacy courses for fostering conceptual learning, literacy and empowerment in university students: Refocusing to conceptual building. Computers in Human Behavior Reports, 7, 100223. https://doi.org/10.1016/j.chbr.2022.100223

Laupichler, M. C., Aster, A., Haverkamp, N., & Raupach, T. (2023). Development of the Scale for the Assessment of Non-Experts' AI Literacy (SNAIL) - An exploratory factor analysis. Computers in Human Behavior Reports, 12, 100338. https://doi.org/10.1016/j.chbr.2023.100338

Laupichler, M. C., Aster, A., & Raupach, T. (2023). Delphi study for the development and preliminary validation of an item set for the assessment of non-experts' AI literacy. Computers and Education: Artificial Intelligence, 4, 100126. https://doi.org/10.1016/j.caeai.2023.100126

Liang, W., Yuksekgonul, M., Mao, Y., Wu, E., & Zou, J. (2023). GPT detectors are biased against non-native English writers. Patterns, 4(7), 100779. https://doi.org/10.1016/j.patter.2023.100779

Ma, Y. (2026). Experiences with artificial intelligence among students at the University of Linyi. International Journal of Education and Humanities, 6(1), 233-251. https://doi.org/10.58557/(ijeh).v6i1.439

Madden, O. N., McKenzie, N., & Daley, J.-L. (2025). Effects of ChatGPT and generative artificial intelligence in higher education: Voices of Jamaican academic faculty. International Journal of Education and Humanities, 5(2), 297-308. https://doi.org/10.58557/(ijeh).v5i2.314

Mansoor, H. M. H., Bawazir, A., Alsabri, M. A., Alharbi, A., & Okela, A. H. (2024). Artificial intelligence literacy among university students - A comparative transnational survey. Frontiers in Communication, 9, 1478476. https://doi.org/10.3389/fcomm.2024.1478476

Nikolić, D., & Basta Nikolić, M. (2026). Designing AI-resilient assessment in higher education: A four-pillar conceptual framework. Frontiers in Artificial Intelligence, 9, 1841682. https://doi.org/10.3389/frai.2026.1841682

Oliveira, M., Zednik, C., Bombaerts, G., Sadowski, B., & Conijn, R. (2025). Assessing students' DRIVE: A framework to evaluate learning through interactions with generative AI. Computers and Education: Artificial Intelligence, 9, 100497. https://doi.org/10.1016/j.caeai.2025.100497

Perkins, M., Furze, L., Roe, J., & MacVaugh, J. (2024). The Artificial Intelligence Assessment Scale (AIAS): A framework for ethical integration of generative AI in educational assessment. Journal of University Teaching & Learning Practice, 21(6). https://doi.org/10.53761/q3azde36

Persada, A. R., & Heryandi, Y. (2026). Instructional design of e-modules and e-courses for AI literacy: Enhancing conceptual, critical, and ethical understanding. International Journal of Education and Humanities, 6(1), 180-193. https://doi.org/10.58557/(ijeh).v6i1.416

Ravšelj, D., Keržič, D., Tomaževič, N., Umek, L., Brezovar, N., Iahad, N. A., ... Aristovnik, A. (2025). Higher education students' perceptions of ChatGPT: A global study of early reactions. PLOS ONE, 20(2), e0315011. https://doi.org/10.1371/journal.pone.0315011

Roe, J., Perkins, M., & Giray, L. (2026). Assessment twins: An approach for strengthening assessment validity in the age of generative AI. Journal of Applied Learning & Teaching, 9(2). https://doi.org/10.37074/jalt.2026.9.2.3

Wani, T. A., Liem, M., Prasad, N., Robinson, K., Nexhip, A., Tassos, M., ... Riley, M. (2026). Susceptibility of assessment types to AI-generated content in digital health and health information management education: Quasi-experimental pilot study. JMIR Medical Education, 12, e82988. https://doi.org/10.2196/82988

Weber-Wulff, D., Anohina-Naumeca, A., Bjelobaba, S., Foltýnek, T., Guerrero-Dib, J., Popoola, O., ... Waddington, L. (2023). Testing of detection tools for AI-generated text. International Journal for Educational Integrity, 19(1), Article 26. https://doi.org/10.1007/s40979-023-00146-z

Winstone, N. E., Gravett, K., & Elkington, S. (2026). Black Box Assessment: Rethinking integrity and learning for a time of generative AI. Assessment & Evaluation in Higher Education. Advance online publication. https://doi.org/10.1080/02602938.2026.2661365

Xia, Q., Weng, X., Ouyang, F., Lin, T. J., & Chiu, T. K. F. (2024). A scoping review on how generative artificial intelligence transforms assessment in higher education. International Journal of Educational Technology in Higher Education, 21(1), Article 40. https://doi.org/10.1186/s41239-024-00468-z

Yilmaz, R., & Karaoglan Yilmaz, F. G. (2023). The effect of generative artificial intelligence (AI)-based tool use on students' computational thinking skills, programming self-efficacy and motivation. Computers and Education: Artificial Intelligence, 4, 100147. https://doi.org/10.1016/j.caeai.2023.100147

Zou, M., & Huang, L. (2023). To use or not to use? Understanding doctoral students' acceptance of ChatGPT in writing through the technology acceptance model. Frontiers in Psychology, 14, 1259531. https://doi.org/10.3389/fpsyg.2023.1259531.

Downloads

Published

2026-10-29

How to Cite

Peng, W. (2026). What counts as evidence of learning when students use generative AI? A critical integrative review of assessment validity in higher education. International Journal of Education and Humanities, 7(3), 647–660. Retrieved from https://i-jeh.com/index.php/ijeh/article/view/517