Elucidating STEM Concepts through Generative AI: A Multi-modal Exploration of Analogical Reasoning

Chen Cao,Zijian Ding,Gyeong-Geon Lee,Jiajun Jiao,Jionghao Lin,Xiaoming Zhai
2023-08-21
Abstract:This study explores the integration of generative artificial intelligence (AI), specifically large language models, with multi-modal analogical reasoning as an innovative approach to enhance science, technology, engineering, and mathematics (STEM) education. We have developed a novel system that utilizes the capacities of generative AI to transform intricate principles in mathematics, physics, and programming into comprehensible metaphors. To further augment the educational experience, these metaphors are subsequently converted into visual form. Our study aims to enhance the learners' understanding of STEM concepts and their learning engagement by using the visual metaphors. We examine the efficacy of our system via a randomized A/B/C test, assessing learning gains and motivation shifts among the learners. Our study demonstrates the potential of applying large language models to educational practice on STEM subjects. The results will shed light on the design of educational system in terms of harnessing AI's potential to empower educational stakeholders.
Artificial Intelligence,Computers and Society,Human-Computer Interaction
What problem does this paper attempt to address?
The problem this paper attempts to address is: how to enhance Science, Technology, Engineering, and Mathematics (STEM) education by leveraging generative artificial intelligence (AI), particularly large language models, in combination with multimodal analogical reasoning. Specifically, the paper aims to tackle the current challenges in STEM education through the following points: 1. **Understanding Abstract Concepts**: Many concepts in the STEM field are abstract and complex, and students often struggle to understand them. The paper proposes a new system that uses generative AI to transform these complex principles into easily understandable metaphors. 2. **Multimodal Learning**: Traditional teaching methods mainly rely on text explanations and code examples, which may not be sufficient to fully explain abstract concepts. The proposed method not only generates textual metaphors but also converts these metaphors into visual forms to further enhance the learning experience. 3. **Increasing Learning Engagement**: By using visual metaphors, the paper hopes to improve students' understanding of STEM concepts and their engagement in learning. The study evaluates the effectiveness of the system through randomized A/B/C testing, examining students' learning gains and changes in motivation. 4. **Innovation in Educational Resources**: The paper explores how to design educational systems using large language models and multimodal learning methods to better utilize the potential of AI, supporting educators and students. Overall, the goal of the paper is to provide an innovative teaching method by combining generative AI and multimodal analogical reasoning to improve the effectiveness of STEM education and student engagement.