Dream-Experiment: A MR User Interface with Natural Multi-channel Interaction for Virtual Experiments
Tianren Luo,Mingmin Zhang,Zhigeng Pan,Zheng Li,Ning Cai,Jinda Miao,Youbin Chen,Mingxi Xu
DOI: https://doi.org/10.1109/tvcg.2020.3023602
IF: 5.2
2020-12-01
IEEE Transactions on Visualization and Computer Graphics
Abstract:This paper studies a set of MR technologies for middle school experimental teaching environments and develops a multi-channel MR user interface called Dream-Experiment. The goal of Dream-Experiment is to improve the traditional MR user interface, so that users can get a real, natural 3D interactive experience like real experiments, but without danger and pollution. In terms of visual presentation, we design multi-camera collaborative registration to realize robust 6-DoF MR interactive space, and also define a complete rendering pipeline to provide improved processing of virtual-real objects' occlusion including translucent devices. In the virtual-real interaction, we provide six interaction modes that support visual interaction, tangible interaction, virtual-real gestures with touching, voice, thermal feeling, and olfactory feeling. After users' testing, we find that Dream-Experiment has better interactive efficiency and user experience than traditional MR environments.
computer science, software engineering