Application of Physical Interactive Mixed Reality System Based on MLAT in the Field of Stage Performance

Yanxiang Zhang,Pengfei Ma,Ali Raja Gulfraz,Li Kong,Xuelian Sun

DOI: https://doi.org/10.1007/978-3-030-25999-0_1

2019-01-01

Abstract:At present, in some real-time occasion like as stage performances, mixed reality effects are often generated by a mechanical means, or only between physical objects and virtual objects. This will lead to many uncontrollable shortcomings. in order to overcome the above disadvantage, the authors developed a mixed reality system for physical interaction. The experiment unfolds between the experiencer and the controllable Four-axis unmanned aerial vehicle with some performance props. The first step of the study is to build the stereoscopic scene through the camera calibration, the target coordinates in image are obtained by infrared sensors deployed on the experimenter and the aircraft respectively by Camshift algorithm and it can be transformed into the spatial coordinates by projection transformation. And the second step is based on different preset working modes and the coordinate relationship expressions between them that have been configured, with the action of the experimenter, Ant Colony Algorithm is used to move the Four-axis UAV to a certain position, thereby realizing this kind of precisely controllable interaction between entities. The final experiment in this paper has proven that our work provided a great mixed reality effect for occasions such as stage performances.