MoiréTracker: Continuous Camera-to-Screen 6-Dof Pose Tracking Based on Moiré Pattern

Jingyi Ning,Lei Xie,Yi Li,Yingying Chen,Yanling Bu,Chuyu Wang,Sanglu Lu,Baoliu Ye
DOI: https://doi.org/10.1109/jsac.2024.3414619
IF: 16.4
2024-01-01
IEEE Journal on Selected Areas in Communications
Abstract:In the realm of AR applications and particularly camera-to-screen interactions, camera tracking stands as a crucial technology. However, the ever-increasing demand for tracking accuracy makes it essential to explore a six-degrees of freedom (6-DoF) tracking technology with ultra-high precision to facilitate micro-motion sensing. In this paper, we propose a novel sensing method MoireTracker to achieve camera's 6-DoF pose tracking with ultra-high precision. MoireTracker outputs camera's continuous 3-DoF trajectory and 3-DoF posture changes according to the captured moir & eacute; patterns, which can be produced by the superposition of camera's Color Filter Array (CFA) and the projection of screen raster on the CFA plane. Thanks to moir & eacute; pattern's high sensitivity to 6-DoF motions, we characterize the relationship between moir & eacute; features and camera's micro pose changes, so as to realize the continuous 6-DoF pose tracking for camera with ultra-high precision. Moreover, our proposal involves a thumbnail-based method aimed at expanding the working range of MoireTracker, enabling the pervasive camera-to-screen interactions. We implement a prototype system and evaluate its performance in real-world environments. Extensive experiment results show that MoireTracker achieves the average trajectory error of 1.20 cm and the posture error of 1.07 degrees.
What problem does this paper attempt to address?