AI-Driven Interface Design for Intelligent Tutoring System Improves Student Engagement

Byungsoo Kim,Hongseok Suh,Jaewe Heo,Youngduck Choi

DOI: https://doi.org/10.48550/arXiv.2009.08976

2020-09-18

Abstract:An Intelligent Tutoring System (ITS) has been shown to improve students' learning outcomes by providing a personalized curriculum that addresses individual needs of every student. However, despite the effectiveness and efficiency that ITS brings to students' learning process, most of the studies in ITS research have conducted less effort to design the interface of ITS that promotes students' interest in learning, motivation and engagement by making better use of AI features. In this paper, we explore AI-driven design for the interface of ITS describing diagnostic feedback for students' problem-solving process and investigate its impacts on their engagement. We propose several interface designs powered by different AI components and empirically evaluate their impacts on student engagement through Santa, an active mobile ITS. Controlled A/B tests conducted on more than 20K students in the wild show that AI-driven interface design improves the factors of engagement by up to 25.13%.

Computers and Society,Human-Computer Interaction

What problem does this paper attempt to address?

### Problems the Paper Aims to Solve

This paper aims to address the inadequacies in the interface design of Intelligent Tutoring Systems (ITS). Although ITS has shown excellent performance in improving student learning outcomes, most ITS research primarily focuses on diagnosing students' knowledge states and recommending appropriate learning items, with less attention given to how optimizing interface design can enhance students' interest, motivation, and engagement. Specifically, this paper explores methods to improve ITS interface design using AI technology and evaluates the impact of these designs on student engagement.

### Research Background

With the rise of online learning, especially during the COVID-19 pandemic, the education system has undergone significant changes, shifting the assessment methods from traditional summative assessments to formative assessments. As an automated teaching service, ITS can effectively address these challenges. However, existing ITS research mostly focuses on diagnosing students' knowledge states and recommending learning items, neglecting the impact of interface design on student outcomes. Therefore, this paper proposes an AI-based interface design method to enhance student engagement.

### Research Methods

1. **Proposing Multiple Interface Designs**: This paper proposes several different interface designs, each incorporating different AI components, providing varying degrees of information and explanations.
2. **Empirical Evaluation**: Through A/B testing in practical applications, the impact of different interface designs on student engagement is evaluated. Over 20,000 students participated in the tests, with evaluation metrics including conversion rate, average revenue per user (ARPU), total profit, and the number of free questions consumed by students.

### Experimental Results

1. **Impact of Radar Charts**: Adding radar charts to page design A improved all engagement metrics. Specifically, the conversion rate increased by 22.68%, ARPU increased by 17.23%, total profit increased by 25.13%, and the number of free questions consumed by students increased by 11.78%.
2. **Comparison of Page Designs A and B**: Page design B performed better than page design A in all engagement metrics. Specifically, the conversion rate increased by 11.07%, ARPU increased by 10.29%, total profit increased by 12.57%, and the number of free questions consumed by students increased by 7.19%.

### Conclusion

This paper empirically demonstrates that optimizing ITS interface design using AI technology can significantly enhance student engagement. Future research directions include further updating and improving interface designs to better utilize and express AI features, thereby further enhancing student engagement.

### Future Work

1. **Updating AI Analysis Page After Learning Sessions**: Updating a summary AI analysis page after diagnostic test learning sessions.
2. **Exploring More Effective Interface Designs**: Exploring more interface design methods to further enhance student engagement.