MindScratch: A Visual Programming Support Tool for Classroom Learning Based on Multimodal Generative AI

Yunnong Chen,Shuhong Xiao,Yaxuan Song,Zejian Li,Lingyun Sun,Liuqing Chen
2024-12-12
Abstract:Programming has become an essential component of K-12 education and serves as a pathway for developing computational thinking skills. Given the complexity of programming and the advanced skills it requires, previous research has introduced user-friendly tools to support young learners. However, our interviews with six programming educators revealed that current tools often fail to reflect classroom learning objectives, offer flexible, high-quality guidance, and foster student creativity. This highlights the need for more adaptive and reflective tools. Therefore, we introduced MindScratch, a multimodal generative AI (GAI) powered visual programming support tool. MindScratch aims to balance structured classroom activities with free programming creation, supporting students in completing creative programming projects based on teacher-set learning objectives while also providing programming scaffolding. Our user study results indicate that, compared to the baseline, MindScratch more effectively helps students achieve high-quality projects aligned with learning objectives. It also enhances students' computational thinking skills and creative thinking. Overall, we believe that GAI-driven educational tools like MindScratch offer students a focused and engaging learning experience.
Human-Computer Interaction
What problem does this paper attempt to address?
The main problem that this paper attempts to solve is that current programming learning tools fail to fully support students in achieving creative programming projects in the classroom environment, especially in the following aspects: 1. **Alignment with Classroom Teaching Goals**: Existing programming tools often cannot reflect the specific goals of classroom learning, resulting in students' learning activities being inconsistent with the goals set by teachers. 2. **Flexible and High - Quality Guidance**: Existing tools lack the ability to provide flexible and high - quality guidance, making it difficult for students to obtain timely and effective help during the programming process. 3. **Stimulating Students' Creativity**: Existing tools fail to fully stimulate students' creativity, limiting students' innovative thinking in programming. Specifically, the paper points out that in programming education, teachers face challenges such as how to ensure that students achieve learning goals in project - based learning, provide personalized creativity and programming support, and make creative resources more accessible to students while being easy to manage for teachers. To address these challenges, the author introduced MindScratch, a visual programming support tool based on multimodal generative AI (GAI), which aims to balance structured classroom activities and free programming creation, help students complete creative programming projects according to the learning goals set by teachers, and provide programming scaffolds. ### Main Contributions 1. **Identifying Challenges through Formative Research**: Through interviews with six programming educators, the challenges in goal alignment, process management, and content delivery in creative programming classrooms, as well as the limitations of existing programming support tools, were determined. 2. **Introducing MindScratch**: A programming learning tool based on interactive mind maps and multimodal generative AI was developed, providing interactive creative exploration, hierarchical code assistance, and creative material generation, thereby enhancing students' learning outcomes in creative classrooms. 3. **Empirical Research**: Through a comparative study, the advantages of MindScratch over the traditional Scratch tool in improving students' programming learning effects were demonstrated, providing design considerations for future creative programming support tools. In summary, MindScratch aims to address the deficiencies of existing programming learning tools in not fully supporting students to achieve creative programming projects in the classroom environment, and provide students with a more structured and creative programming learning experience.