Abstract:Time-of-flight (ToF) imaging has become a widespread technique for depth estimation, allowing affordable off-the-shelf cameras to provide depth maps in real time. However, multipath interference (MPI) resulting from indirect illumination significantly degrades the captured depth. Most previous works have tried to solve this problem by means of complex hardware modifications or costly computations. In this work we avoid these approaches, and propose a new technique that corrects errors in depth caused by MPI that requires no camera modifications, and corrects depth in just 10 milliseconds per frame. By observing that most MPI information can be expressed as a function of the captured depth, we pose MPI removal as a convolutional approach, and model it using a convolutional neural network. In particular, given that the input and output data present similar structure, we base our network in an autoencoder, which we train in two stages: first, we use the encoder (convolution filters) to learn a suitable basis to represent corrupted range images; then, we train the decoder (deconvolution filters) to correct depth from the learned basis from synthetically generated scenes. This approach allows us to tackle the lack of reference data, by using a large-scale captured training set with corrupted depth to train the encoder, and a smaller synthetic training set with ground truth depth to train the corrector stage of the network, which we generate by using a physically-based, time-resolved rendering. We demonstrate and validate our method on both synthetic and real complex scenarios, using an off-the-shelf ToF camera, and with only the captured incorrect depth as input.

Depth Errors Analysis and Correction for Time-of-Flight (tof) Cameras.

DoCam: Depth Sensing with an Optical Image Stabilization Supported RGB Camera.

Data Processing Method of Time-of-flight 3D Imaging Camera

Iterative Error Removal for Time-of-Flight Depth Imaging

Accurate 3D reconstruction using a multi-phase ToF camera

Multi-Camera Interference Cancellation of Time-Of-Flight (tof) Cameras

Reliable Fusion of ToF and Stereo Data Based on Joint Depth Filter

All Pixels Calibration for ToF Camera

DeepToF: Off-the-Shelf Real-Time Correction of Multipath Interference in Time-of-Flight Imaging

Real-Time Artifact Compensation for Depth Images of Multi-Frequency ToF

A Learning Method To Optimize Depth Accuracy And Frame Rate For Time Of Flight Camera

K-nearest Neighborhood Based Integration of Time-of-flight Cameras and Passive Stereo for High-Accuracy Depth Maps.

Study of the depth accuracy and entropy characteristics of a ToF camera with coupled noise

Restoration of Motion Blur in Time-of-Flight Depth Image Using Data Alignment.

K-nearestneighborhood Basedintegrationoftime-of-flight Camerasandpassivestereoforhigh-Accuracydepthmaps ∗

An Overview of Depth Cameras and Range Scanners Based on Time-of-Flight Technologies

Calibration and Error Compensation With Time of Flight 3D Camera

TOF Depth Map Super-resolution Using Compressive Sensing

DELTAR: Depth Estimation from a Light-Weight ToF Sensor and RGB Image

Assessment and Improvement of Distance Measurement Accuracy for Time-of-Flight Cameras

High quality depth maps from stereo matching and ToF camera.