Abstract:Contribution: This research explores the benefits and challenges of developing, deploying, and evaluating a large language models (LLMs) chatbot, MoodleBot, in computer science classroom settings. It highlights the potential of integrating LLMs into LMSs like Moodle to support self-regulated learning (SRL) and help-seeking behavior. Background: Computer science educators face immense challenges incorporating novel tools into LMSs to create a supportive and engaging learning environment. MoodleBot addresses this challenge by offering an interactive platform for both students and teachers. Research Questions: Despite issues like bias, hallucinations, and teachers’ and educators’ resistance to embracing new (AI) technologies, this research investigates two questions: (RQ1) To what extent do students accept MoodleBot as a valuable tool for learning support? (RQ2) How accurately does MoodleBot churn out responses, and how congruent are these with the established course content? Methodology: This study reviews pedagogical literature on AI-driven chatbots and adopts the retrieval-augmented generation (RAG) approach for MoodleBot’s design and data processing. The technology acceptance model (TAM) evaluates user acceptance through constructs like perceived usefulness (PU) and Ease of Use. Forty-six students participated, with 30 completing the TAM questionnaire. Findings: LLM-based chatbots like MoodleBot can significantly improve the teaching and learning process. This study revealed a high accuracy rate (88%) in providing course-related assistance. Positive responses from students attest to the efficacy and applicability of AI-driven educational tools. These findings indicate that educational chatbots are suitable for integration into courses to improve personalized learning and reduce teacher administrative burden, although improvements in automated fact-checking are needed.

It Takes a Village: Integrating an Adaptive Chatbot into an Online Gaming Community

Teaching Machines to Converse

AI Chatbots as Multi-Role Pedagogical Agents: Transforming Engagement in CS Education

It Takes a Village: A Distributed Training Model for AI-based Chatbots

From Eliza to XiaoIce: Challenges and Opportunities with Social Chatbots

When design workshops meet chatbots: Meaningful participation at scale?

Enhanced engineering design behaviour using chatbots with user experience

Adolescent-Centric Design of an Online Safety Chatbot

A Piece of Theatre: Investigating How Teachers Design LLM Chatbots to Assist Adolescent Cyberbullying Education

Using Chatbots to Teach Languages

Adapting Self-Regulated Learning in an Age of Generative Artificial Intelligence Chatbots

Exploring diversity perceptions in a community through a Q&A chatbot

The Design and Implementation of XiaoIce, an Empathetic Social Chatbot

The HCI Aspects of Public Deployment of Research Chatbots: A User Study, Design Recommendations, and Open Challenges

Fictional Worlds, Real Connections: Developing Community Storytelling Social Chatbots through LLMs

"Can I be More Social with a Chatbot?": Social Connectedness Through Interactions of Autistic Adults with a Conversational Virtual Human

An LLM-Driven Chatbot in Higher Education for Databases and Information Systems

How should my chatbot interact? A survey on human-chatbot interaction design

Patterns of Sociotechnical Design Preferences of Chatbots for Intergenerational Collaborative Innovation : A Q Methodology Study

Multi-Purpose NLP Chatbot : Design, Methodology & Conclusion

Educational Psychology Aspects of Learning with Chatbots without Artificial Intelligence: Suggestions for Designers