A novel animation authoring framework for the virtual teacher performing experiment in mixed reality
Zhenning Zhang,Yihe Wu,Zhigeng Pan,Weiqing Li,Zhiyong Su
DOI: https://doi.org/10.1002/cae.22472
2021-11-10
Computer Applications in Engineering Education
Abstract:Abstract Virtual teachers embedded into reality enable the transfer of skills and knowledge to students by demonstration. However, authoring the animation of virtual characters aligned with real environments is difficult for educators. In this study, a novel animation authoring framework for demonstration teaching is proposed based on human–computer interaction and intelligent algorithm, and its application is demonstrated in mixed reality experimental education. We developed a simulation experiment environment; users could operate equipment using gestures by a Leap Motion controller. The interaction trajectory can be adaptively adjusted by the dynamic motion primitives algorithm to generate a virtual teacher demonstration trajectory aligned with real environments. For the details of the virtual teacher animation, the deep reinforcement learning algorithm performed the adaptive grabbing for objects of different shapes and sizes. Our experimental results showed that the framework is easy to use; users are able to easily author a natural animation of a virtual teacher performing a chemical experiment aligned with real environments. Also, the user feedback showed that the virtual teacher's demonstration animation in MR is impressive, and students can learn from the animation by observation and imitation.
computer science, interdisciplinary applications,engineering, multidisciplinary,education, scientific disciplines