Single-shot 3D Tracking Based on Polarization Multiplexed Fourier-phase Camera
Jiajie Teng,Chengyang Hu,Honghao Huang,Minghua Chen,Sigang Yang,Hongwei Chen
DOI: https://doi.org/10.1364/prj.432292
IF: 7.6
2021-01-01
Photonics Research
Abstract:For moving objects, 3D mapping and tracking has found important applications in the 3D reconstruction for vision odometry or simultaneous localization and mapping. This paper presents a novel camera architecture to locate the fast-moving objects in four-dimensional (4D) space ( x , y , z , t ) through a single-shot image. Our 3D tracking system records two orthogonal fields-of-view (FoVs) with different polarization states on one polarization sensor. An optical spatial modulator is applied to build up temporal Fourier-phase coding channels, and the integration is performed in the corresponding CMOS pixels during the exposure time. With the 8 bit grayscale modulation, each coding channel can achieve 256 times temporal resolution improvement. A fast single-shot 3D tracking system with 0.78 ms temporal resolution in 200 ms exposure is experimentally demonstrated. Furthermore, it provides a new image format, Fourier-phase map, which has a compact data volume. The latent spatio-temporal information in one 2D image can be efficiently reconstructed at relatively low computation cost through the straightforward phase matching algorithm. Cooperated with scene-driven exposure as well as reasonable Fourier-phase prediction, one could acquire 4D data ( x , y , z , t ) of the moving objects, segment 3D motion based on temporal cues, and track targets in a complicated environment.