Abstract:Depth cameras are emerging as a cornerstone modality with diverse applications that directly or indirectly rely on measured depth, including personal devices, robotics, and self-driving vehicles. Although time-of-flight (ToF) methods have fueled these applications, the precision and robustness of ToF methods is limited by relying on photon time-tagging or modulation after photo-conversion. Successful optical modulation approaches have been restricted fiber-coupled modulation with large coupling losses or interferometric modulation with sub-cm range, and the precision gap between interferometric methods and ToF methods is more than three orders of magnitudes. In this work, we close this gap and propose a computational imaging method for all-optical free-space correlation before photo-conversion that achieves micron-scale depth resolution with robustness to surface reflectance and ambient light with conventional silicon intensity sensors. To this end, we solve two technical challenges: modulating at GHz rates and computational phase unwrapping. We propose an imaging approach with resonant polarization modulators and devise a novel optical dual-pass frequency-doubling which achieves high modulation contrast at more than 10GHz. At the same time, centimeter-wave modulation together with a small modulation bandwidth render existing phase unwrapping methods ineffective. We tackle this problem with a neural phase unwrapping method that exploits that adjacent wraps are often highly correlated. We validate the proposed method in simulation and experimentally, where it achieves micron-scale depth precision. We demonstrate precise depth sensing independently of surface texture and ambient light and compare against existing analog demodulation methods, which we outperform across all tested scenarios.

Simultaneous Estimation of Object Region and Depth in Participating Media Using a ToF Camera

Defogging Kinect: Simultaneous Estimation of Object Region and Depth in Foggy Scenes

DoCam: Depth Sensing with an Optical Image Stabilization Supported RGB Camera.

Reliable Fusion of ToF and Stereo Data Based on Joint Depth Filter

K-nearest Neighborhood Based Integration of Time-of-flight Cameras and Passive Stereo for High-Accuracy Depth Maps.

TOF Depth Map Super-resolution Using Compressive Sensing

DELTAR: Depth Estimation from a Light-Weight ToF Sensor and RGB Image

K-nearestneighborhood Basedintegrationoftime-of-flight Camerasandpassivestereoforhigh-Accuracydepthmaps ∗

Iterative Error Removal for Time-of-Flight Depth Imaging

Depth Estimation Algorithm for Light Field Data by Epipolar Image Analysis and Region Interpolation

Low Power Depth Estimation of Rigid Objects for Time-of-Flight Imaging

Accurate 3D reconstruction using a multi-phase ToF camera

High quality depth maps from stereo matching and ToF camera.

Time-of-Flight (ToF) Cameras for Underwater Situational Awareness

Itof-Flow-based High Frame Rate Depth Imaging

3D Reconstruction Using 3D Registration-Based ToF-Stereo Fusion

Spatio-Temporal Depth Recovery of Dynamic Scenes with Multiple Handheld Cameras

Single-shot ToF sensing with sub-mm precision using conventional CMOS sensors

Simultaneous Multifrequency Demodulation for Single-Shot Multiple-Path ToF Imaging

Centimeter-Wave Free-Space Time-of-Flight Imaging

Single-photon 3D imaging with deep sensor fusion