Abstract:Deep neural networks are increasingly employed to model classroom dialogue and provide teachers with prompt and valuable feedback on their teaching practices. However, these deep learning models often have intricate structures with numerous unknown parameters, functioning as black boxes. The lack of clear explanations regarding their classroom dialogue analysis likely leads teachers to distrust and underutilize these AI‐powered models. To tackle this issue, we leveraged explainable AI to unravel classroom dialogue analysis and conducted an experiment to evaluate the effects of explanations. Fifty‐nine pre‐service teachers were recruited and randomly assigned to either a treatment (n = 30) or control (n = 29) group. Initially, both groups learned to analyse classroom dialogue using AI‐powered models without explanations. Subsequently, the treatment group received both AI analysis and explanations, while the control group continued to receive only AI predictions. The results demonstrated that teachers in the treatment group exhibited significantly higher levels of trust in and technology acceptance of AI‐powered models for classroom dialogue analysis compared to those in the control group. Notably, there were no significant differences in cognitive load between the two groups. Furthermore, teachers in the treatment group expressed high satisfaction with the explanations. During interviews, they also elucidated how the explanations changed their perceptions of model features and attitudes towards the models. This study is among the pioneering works to propose and validate the use of explainable AI to address interpretability challenges within deep learning‐based models in the context of classroom dialogue analysis.Practitioner notesWhat is already known about this topic Classroom dialogue is recognized as a crucial element in the teaching and learning process. Researchers have increasingly utilized AI techniques, particularly deep learning methods, to analyse classroom dialogue. Deep learning‐based models, characterized by their intricate structures, often function as black boxes, lacking the ability to provide transparent explanations regarding their analysis. This limitation can result in teachers harbouring distrust and underutilizing these models. What this paper adds This paper highlights the importance of incorporating explainable AI approaches to tackle the interpretability issues associated with deep learning‐based models utilized for classroom dialogue analysis. Through an experimental study, this paper demonstrates that providing model explanations enhances teachers' trust in and technology acceptance of AI‐powered classroom dialogue models, without increasing their cognitive load. Teachers express satisfaction with the model explanations provided by explainable AI. Implications for practice and/or policy The integration of explainable AI can effectively address the challenge of interpretability in complex AI‐powered models used for analysing classroom dialogue. Intelligent teaching systems designed for classroom dialogue can benefit from advanced AI models and explainable AI approaches, which offer users both automated analysis and clear explanations. By enabling users to understand the underlying rationale behind the analysis, the explanations can contribute to fostering trust and acceptance of the AI models among users.

Need of AI in Modern Education: in the Eyes of Explainable AI (xAI)

Explainable Artificial Intelligence in education

Explainable artificial intelligence for education and training

Human-Centric eXplainable AI in Education

Can Explainable AI Explain Unfairness? A Framework for Evaluating Explainable AI

Explainable AI (XAI): History, Basic Ideas and Methods

Explainable Artificial Intelligence Approaches: A Survey

AI in Education: Bridging the Divide or Widening the Gap? Exploring Equity, Opportunities, and Challenges in the Digital Age

Unpacking the "Black Box" of AI in Education

Using explainable AI to unravel classroom dialogue analysis: Effects of explanations on teachers' trust, technology acceptance and cognitive load

FairAIED: Navigating Fairness, Bias, and Ethics in Educational AI Applications

Survey on Explainable AI: From Approaches, Limitations and Applications Aspects

Explainability of Artificial Intelligence Methods, Applications and Challenges: A Comprehensive Survey

Advancing Educational Insights: Explainable AI Models for Informed Decision-Making

Towards Fair and Explainable AI using a Human-Centered AI Approach

Fairness, Bias, and Ethics in AI: Exploring the Factors Affecting Student Performance

Explainable Artificial Intelligence: Concepts, Applications, Research Challenges and Visions

Equity and Artificial Intelligence in Education: Will "AIEd" Amplify or Alleviate Inequities in Education?

Exploring the role of AI in education

The Dilemma and Countermeasures of AI in Educational Application

Unveiling the shadows: Beyond the hype of AI in education