Abstract:Contribution: This research explores the benefits and challenges of developing, deploying, and evaluating a large language models (LLMs) chatbot, MoodleBot, in computer science classroom settings. It highlights the potential of integrating LLMs into LMSs like Moodle to support self-regulated learning (SRL) and help-seeking behavior. Background: Computer science educators face immense challenges incorporating novel tools into LMSs to create a supportive and engaging learning environment. MoodleBot addresses this challenge by offering an interactive platform for both students and teachers. Research Questions: Despite issues like bias, hallucinations, and teachers’ and educators’ resistance to embracing new (AI) technologies, this research investigates two questions: (RQ1) To what extent do students accept MoodleBot as a valuable tool for learning support? (RQ2) How accurately does MoodleBot churn out responses, and how congruent are these with the established course content? Methodology: This study reviews pedagogical literature on AI-driven chatbots and adopts the retrieval-augmented generation (RAG) approach for MoodleBot’s design and data processing. The technology acceptance model (TAM) evaluates user acceptance through constructs like perceived usefulness (PU) and Ease of Use. Forty-six students participated, with 30 completing the TAM questionnaire. Findings: LLM-based chatbots like MoodleBot can significantly improve the teaching and learning process. This study revealed a high accuracy rate (88%) in providing course-related assistance. Positive responses from students attest to the efficacy and applicability of AI-driven educational tools. These findings indicate that educational chatbots are suitable for integration into courses to improve personalized learning and reduce teacher administrative burden, although improvements in automated fact-checking are needed.

Walert: Putting Conversational Search Knowledge into Action by Building and Evaluating a Large Language Model-Powered Chatbot

A Complete Survey on LLM-based AI Chatbots

An LLM-Driven Chatbot in Higher Education for Databases and Information Systems

Beyond ChatBots: ExploreLLM for Structured Thoughts and Personalized Model Responses

Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference

Framework to enable and test conversational assistant for APIs and RPAs

Bio-Eng-LMM AI Assist chatbot: A Comprehensive Tool for Research and Education

Conversational Challenges in AI-Powered Data Science: Obstacles, Needs, and Design Opportunities

Learnings from a Large-Scale Deployment of an LLM-Powered Expert-in-the-Loop Healthcare Chatbot

Modular Conversational Agents for Surveys and Interviews

EduChat: A Large-Scale Language Model-based Chatbot System for Intelligent Education

Developing a Retrieval Augmented Generation (RAG) Chatbot App Using Adaptive Large Language Models (LLM) and LangChain Framework

Hello Again! LLM-powered Personalized Agent for Long-term Dialogue

Beyond Traditional Teaching: The Potential of Large Language Models and Chatbots in Graduate Engineering Education

Leveraging LLM: Implementing an Advanced AI Chatbot for Healthcare

From Questions to Insightful Answers: Building an Informed Chatbot for University Resources

How to Align Large Language Models for Teaching English? Designing and Developing LLM based-Chatbot for Teaching English Conversation in EFL, Findings and Limitations

Navigating the Unknown: A Chat-Based Collaborative Interface for Personalized Exploratory Tasks

Building a Legal Dialogue System: Development Process, Challenges and Opportunities

Chat with the Environment: Interactive Multimodal Perception Using Large Language Models