Abstract:Deep neural networks are increasingly employed to model classroom dialogue and provide teachers with prompt and valuable feedback on their teaching practices. However, these deep learning models often have intricate structures with numerous unknown parameters, functioning as black boxes. The lack of clear explanations regarding their classroom dialogue analysis likely leads teachers to distrust and underutilize these AI‐powered models. To tackle this issue, we leveraged explainable AI to unravel classroom dialogue analysis and conducted an experiment to evaluate the effects of explanations. Fifty‐nine pre‐service teachers were recruited and randomly assigned to either a treatment (n = 30) or control (n = 29) group. Initially, both groups learned to analyse classroom dialogue using AI‐powered models without explanations. Subsequently, the treatment group received both AI analysis and explanations, while the control group continued to receive only AI predictions. The results demonstrated that teachers in the treatment group exhibited significantly higher levels of trust in and technology acceptance of AI‐powered models for classroom dialogue analysis compared to those in the control group. Notably, there were no significant differences in cognitive load between the two groups. Furthermore, teachers in the treatment group expressed high satisfaction with the explanations. During interviews, they also elucidated how the explanations changed their perceptions of model features and attitudes towards the models. This study is among the pioneering works to propose and validate the use of explainable AI to address interpretability challenges within deep learning‐based models in the context of classroom dialogue analysis.Practitioner notesWhat is already known about this topic Classroom dialogue is recognized as a crucial element in the teaching and learning process. Researchers have increasingly utilized AI techniques, particularly deep learning methods, to analyse classroom dialogue. Deep learning‐based models, characterized by their intricate structures, often function as black boxes, lacking the ability to provide transparent explanations regarding their analysis. This limitation can result in teachers harbouring distrust and underutilizing these models. What this paper adds This paper highlights the importance of incorporating explainable AI approaches to tackle the interpretability issues associated with deep learning‐based models utilized for classroom dialogue analysis. Through an experimental study, this paper demonstrates that providing model explanations enhances teachers' trust in and technology acceptance of AI‐powered classroom dialogue models, without increasing their cognitive load. Teachers express satisfaction with the model explanations provided by explainable AI. Implications for practice and/or policy The integration of explainable AI can effectively address the challenge of interpretability in complex AI‐powered models used for analysing classroom dialogue. Intelligent teaching systems designed for classroom dialogue can benefit from advanced AI models and explainable AI approaches, which offer users both automated analysis and clear explanations. By enabling users to understand the underlying rationale behind the analysis, the explanations can contribute to fostering trust and acceptance of the AI models among users.

Towards transparent and trustworthy prediction of student learning achievement by including instructors as co-designers: a case study

Course Success Prediction and Early Identification of At-Risk Students Using Explainable Artificial Intelligence

Explainable AI for Cheating Detection and Churn Prediction in Online Games

Explainable Artificial Intelligence in education

Integration of artificial intelligence performance prediction and learning analytics to improve student learning in online engineering course

Using explainable AI to unravel classroom dialogue analysis: Effects of explanations on teachers' trust, technology acceptance and cognitive load

A Study of Academic Achievement Attribution Analysis Based on Explainable Machine Learning Techniques.

Are Good Explainers Secretly Human-in-the-Loop Active Learners?

Advancing Educational Insights: Explainable AI Models for Informed Decision-Making

Interpretable Narrative Explanation for ML Predictors with LP: A Case Study for XAI

A Prescriptive Learning Analytics Framework: Beyond Predictive Modelling and onto Explainable AI with Prescriptive Analytics and ChatGPT

Explainable Machine Learning Prediction for the Academic Performance of Deaf Scholars

Interpretable Dropout Prediction: Towards XAI-Based Personalized Intervention

Explainable artificial intelligence for education and training

Explainable Active Learning (XAL): An Empirical Study of How Local Explanations Impact Annotator Experience

Transferring Domain Knowledge with (X)AI-Based Learning Systems

Confident Teacher, Confident Student? A Novel User Study Design for Investigating the Didactic Potential of Explanations and their Impact on Uncertainty

Explainable AI as a Social Microscope: A Case Study on Academic Performance

Will We Trust What We Don't Understand? Impact of Model Interpretability and Outcome Feedback on Trust in AI

Model Interpretation and Explainability: Towards Creating Transparency in Prediction Models

On Human Predictions with Explanations and Predictions of Machine Learning Models: A Case Study on Deception Detection