Higher education assessment practice in the era of generative AI tools

Bayode Ogunleye,Kudirat Ibilola Zakariyyah,Oluwaseun Ajao,Olakunle Olayinka,Hemlata Sharma

DOI: https://doi.org/10.37074/jalt.2024.7.1.28

2024-04-01

Abstract:The higher education (HE) sector benefits every nation's economy and society at large. However, their contributions are challenged by advanced technologies like generative artificial intelligence (GenAI) tools. In this paper, we provide a comprehensive assessment of GenAI tools towards assessment and pedagogic practice and, subsequently, discuss the potential impacts. This study experimented using three assessment instruments from data science, data analytics, and construction management disciplines. Our findings are two-fold: first, the findings revealed that GenAI tools exhibit subject knowledge, problem-solving, analytical, critical thinking, and presentation skills and thus can limit learning when used unethically. Secondly, the design of the assessment of certain disciplines revealed the limitations of the GenAI tools. Based on our findings, we made recommendations on how AI tools can be utilised for teaching and learning in HE.

Information Retrieval,Artificial Intelligence,Computer Vision and Pattern Recognition,Machine Learning

What problem does this paper attempt to address?

This paper aims to explore the impact and potential applications of generative artificial intelligence (GenAI) tools in higher education assessment practices. Specifically, the paper evaluates the performance of GenAI tools in the fields of data science, data analysis, and construction management through experiments and draws the following main conclusions:

1. **Capabilities of GenAI Tools**: The study shows that GenAI tools such as ChatGPT and Gemini possess certain professional knowledge, problem-solving abilities, analytical skills, critical thinking abilities, and presentation skills. These skills are crucial for teaching and assessment.

2. **Importance of Ethical Use**: When these tools are used unethically in assessments, they may limit students' learning and development. For example, in the fields of data science and data analysis, if students rely on these tools to cheat, it may affect their actual skill development.

3. **Interdisciplinary Differences**: The study found that GenAI tools perform differently across various disciplines. In the fields of data science and data analysis, these tools perform well, but in the field of construction management, they perform poorly. This is mainly due to the varying availability of online learning resources in different disciplines, leading to differences in the amount of training data.

Based on these findings, the paper proposes the following recommendations:
- Modify curricula to adapt to the changing skill demands.
- Provide appropriate guidance to regulate the use of GenAI tools to avoid negative impacts on students' learning experiences.
- Emphasize that assessment design should consider ethics and complexity to ensure comprehensive student development.