MoiréVision: A Generalized Moiré-based Mechanism for 6-Dof Motion Sensing

Jingyi Ning,Lei Xie,Zhihao Yan,Yanling Bu,Jun Luo

DOI: https://doi.org/10.1145/3636534.3649374

2024-01-01

Abstract:Ultra-high precision motion sensing leveraging computer vision (CV) is a key technology in many high-precision AR/VR applications such as precise industrial manufacture and image-guided surgery, yet conventional CV can be challenged by moire-based sensing mechanism, thanks to moire pattern's high sensitivity to six degrees of freedom (6-DoF) pose changes. Unfortunately, existing moire-based solutions, in their infancy, cannot deal with complicated curvilinear moire patterns caused by various perspective angles. In this paper, we propose a generalized moire-based mechanism, MoireVision, towards practical adoptions; it relies on high-frequency gratings as visual marker to help extract the fine-grained feature points for ultra-high precision motion sensing. As the foundation of general moire-based sensing, we propose a formulation to characterize "uncontrolled" curvilinear moire patterns in practical scenarios. To deal with the problem of moire feature interference in practice, we propose a Gabor-based algorithm to separate overlapped curvilinear moire patterns from two dimensions. Furthermore, to extract fine-grained feature points for high-precision motion sensing, we propose a bending function-based model and a resolution-enhanced strategy to reconstruct detailed texture of moire markers and extract moire feature points at sub-pixel level. Extensive experimental results show that MoireVision greatly enhances the usability and generalizability of moire-based sensing systems in real-world applications.