AI Chatbots as Multi-Role Pedagogical Agents: Transforming Engagement in CS Education

Cassie Chen Cao,Zijian Ding,Jionghao Lin,Frank Hopfgartner

2023-08-08

Abstract:This study investigates the use of Artificial Intelligence (AI)-powered, multi-role chatbots as a means to enhance learning experiences and foster engagement in computer science education. Leveraging a design-based research approach, we develop, implement, and evaluate a novel learning environment enriched with four distinct chatbot roles: Instructor Bot, Peer Bot, Career Advising Bot, and Emotional Supporter Bot. These roles, designed around the tenets of Self-Determination Theory, cater to the three innate psychological needs of learners - competence, autonomy, and relatedness. Additionally, the system embraces an inquiry-based learning paradigm, encouraging students to ask questions, seek solutions, and explore their curiosities. We test this system in a higher education context over a period of one month with 200 participating students, comparing outcomes with conditions involving a human tutor and a single chatbot. Our research utilizes a mixed-methods approach, encompassing quantitative measures such as chat log sequence analysis, and qualitative methods including surveys and focus group interviews. By integrating cutting-edge Natural Language Processing techniques such as topic modelling and sentiment analysis, we offer an in-depth understanding of the system's impact on learner engagement, motivation, and inquiry-based learning. This study, through its rigorous design and innovative approach, provides significant insights into the potential of AI-empowered, multi-role chatbots in reshaping the landscape of computer science education and fostering an engaging, supportive, and motivating learning environment.

Artificial Intelligence

What problem does this paper attempt to address?

The paper primarily explores how to utilize AI-driven multi-role chatbots to improve student engagement, learning motivation, and personalized learning experiences in computer science (CS) education. Specifically, the research aims to address the following issues:

1. **Enhancing Student Engagement and Motivation**: To tackle the problems of low participation rates, high student-to-teacher ratios, and students' reluctance to engage in inquiry-based learning in current CS education, the study aims to stimulate students' interest and motivation by introducing multi-role chatbots.

2. **Personalized Learning Experience**: Given that traditional CS course content is abstract and lacks personalized attention and feedback, the research proposes meeting students' personalized needs through different roles of chatbots, including academic guidance, career planning advice, social support, and emotional support.

3. **Optimizing Chatbot Design**: Considering that the interaction quality of chatbots is crucial to their acceptance and learning effectiveness, the research also aims to optimize the design and role allocation of multi-role chatbots to ensure they can effectively contribute to creating a vibrant and supportive learning environment.

To achieve these goals, the research team designed a learning environment that includes four different role chatbots and evaluated their impact on students' learning experiences through a series of experimental methods. Additionally, the research proposed a system architecture that details the functions of various chatbots and their collaboration methods to support the learning process. Ultimately, the study demonstrated the potential of large language model (LLM)-driven chatbots in enhancing student engagement and providing personalized support through preliminary results.