Aditya Gunturu,Yi Wen,Nandi Zhang,Jarin Thundathil,Rubaiat Habib Kazi,Ryo Suzuki
Abstract:We introduce Augmented Physics, a machine learning-integrated authoring tool designed for creating embedded interactive physics simulations from static textbook diagrams. Leveraging recent advancements in computer vision, such as Segment Anything and Multi-modal LLMs, our web-based system enables users to semi-automatically extract diagrams from physics textbooks and generate interactive simulations based on the extracted content. These interactive diagrams are seamlessly integrated into scanned textbook pages, facilitating interactive and personalized learning experiences across various physics concepts, such as optics, circuits, and kinematics. Drawing from an elicitation study with seven physics instructors, we explore four key augmentation strategies: 1) augmented experiments, 2) animated diagrams, 3) bi-directional binding, and 4) parameter visualization. We evaluate our system through technical evaluation, a usability study (N=12), and expert interviews (N=12). Study findings suggest that our system can facilitate more engaging and personalized learning experiences in physics education.
Human-Computer Interaction,Computer Vision and Pattern Recognition,Machine Learning
What problem does this paper attempt to address?
### Problems the Paper Attempts to Solve
This paper attempts to address the issue in physics education where static textbook diagrams fail to provide a dynamic, interactive learning experience. Specifically:
1. **Limitations of Static Diagrams**:
- While static diagrams can illustrate simple concepts, they become unintuitive for systems involving motion or changes over time. For example, a gravitational potential energy diagram cannot show the forces acting on an object at different time points, and the motion of celestial bodies in elliptical orbits cannot be depicted through static diagrams to show changes in velocity.
2. **Lack of Video Resources**:
- Although videos can enhance understanding, they lack interactivity and experimental opportunities. Students cannot conduct interactive experiments through videos, and such interaction is crucial for building intuitive understanding.
3. **Insufficiencies of Existing Simulation Tools**:
- Existing online simulation tools often require students to create simulations from scratch, assuming they already have a deep understanding of the topic. This can lead to confusion at the outset. In contrast, existing diagrams in textbooks provide guided steps that help in understanding the topic.
4. **Mismatch and Distraction from External Resources**:
- When students seek external resources to supplement classroom materials, they may encounter content that is inconsistent with classroom teaching, and deviating from the main material can lead to distraction. Enhancing the existing static diagrams in textbooks can directly address these issues.
### Solution
To address the above issues, the authors propose **Augmented Physics**, an integrated authoring tool with machine learning capabilities designed to transform static textbook diagrams into embedded interactive physics simulations. The specific methods include:
1. **Enhanced Experiments**:
- Dynamically simulate diagrams based on physical principles, allowing students to interact with concepts in the textbook and observe experimental results through real-time feedback. For example, observing changes in light paths when altering the position of a lens, or adjusting mass and velocity parameters in a collision simulation.
2. **Animated Diagrams**:
- Convert static diagrams into looping dynamic animations to show processes that change over time. For example, in acoustics, displaying continuous sound waves and electromagnetic wave motion to enhance understanding.
3. **Bidirectional Binding**:
- Physics systems often involve bidirectional relationships between measurable parameters. For example, the relationship between kinetic and potential energy, or between mass and acceleration. By enhancing static diagrams, these parameters can be manipulated bidirectionally. For instance, adjusting the height of a ball and observing changes in a bar graph, and vice versa.
4. **Parameter Visualization**:
- Create data visualizations for diagrams lacking accompanying visual data. For example, a digital oscilloscope in a circuit diagram can measure the voltage between any two points in the circuit; a velocity-time graph on a planet's elliptical orbit can show the speed changes at perihelion.
Through these methods, Augmented Physics aims to provide a richer and more personalized physics learning experience, enabling students to interact and experiment directly on textbook pages without the need to seek external resources.