Abstract:Contribution: This research explores the benefits and challenges of developing, deploying, and evaluating a large language models (LLMs) chatbot, MoodleBot, in computer science classroom settings. It highlights the potential of integrating LLMs into LMSs like Moodle to support self-regulated learning (SRL) and help-seeking behavior. Background: Computer science educators face immense challenges incorporating novel tools into LMSs to create a supportive and engaging learning environment. MoodleBot addresses this challenge by offering an interactive platform for both students and teachers. Research Questions: Despite issues like bias, hallucinations, and teachers’ and educators’ resistance to embracing new (AI) technologies, this research investigates two questions: (RQ1) To what extent do students accept MoodleBot as a valuable tool for learning support? (RQ2) How accurately does MoodleBot churn out responses, and how congruent are these with the established course content? Methodology: This study reviews pedagogical literature on AI-driven chatbots and adopts the retrieval-augmented generation (RAG) approach for MoodleBot’s design and data processing. The technology acceptance model (TAM) evaluates user acceptance through constructs like perceived usefulness (PU) and Ease of Use. Forty-six students participated, with 30 completing the TAM questionnaire. Findings: LLM-based chatbots like MoodleBot can significantly improve the teaching and learning process. This study revealed a high accuracy rate (88%) in providing course-related assistance. Positive responses from students attest to the efficacy and applicability of AI-driven educational tools. These findings indicate that educational chatbots are suitable for integration into courses to improve personalized learning and reduce teacher administrative burden, although improvements in automated fact-checking are needed.

ProxyLLM : LLM-Driven Framework for Customer Support Through Text-Style Transfer

Building Emotional Support Chatbots in the Era of LLMs

SweetieChat: A Strategy-Enhanced Role-playing Framework for Diverse Scenarios Handling Emotional Support Agent

How to Align Large Language Models for Teaching English? Designing and Developing LLM based-Chatbot for Teaching English Conversation in EFL, Findings and Limitations

LLM-based Smart Reply (LSR): Enhancing Collaborative Performance with ChatGPT-mediated Smart Reply System

Beyond ChatBots: ExploreLLM for Structured Thoughts and Personalized Model Responses

Interactive Agents: Simulating Counselor-Client Psychological Counseling via Role-Playing LLM-to-LLM Interactions

Context-Aware LLM Translation System Using Conversation Summarization and Dialogue History

ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debate

An LLM-Driven Chatbot in Higher Education for Databases and Information Systems

CHOPS: CHat with custOmer Profile Systems for Customer Service with LLMs

Understanding the Benefits and Challenges of Deploying Conversational AI Leveraging Large Language Models for Public Health Intervention

Task Supportive and Personalized Human-Large Language Model Interaction: A User Study

Does chat change LLM's mind? Impact of Conversation on Psychological States of LLMs

Beyond Text: Implementing Multimodal Large Language Model-Powered Multi-Agent Systems Using a No-Code Platform

The Typing Cure: Experiences with Large Language Model Chatbots for Mental Health Support

Enhancing user experience and trust in advanced LLM-based conversational agents

Understanding the Therapeutic Relationship between Counselors and Clients in Online Text-based Counseling using LLMs

Understanding Large-Language Model (LLM)-powered Human-Robot Interaction