Abstract:Time-of-flight (ToF) imaging has become a widespread technique for depth estimation, allowing affordable off-the-shelf cameras to provide depth maps in real time. However, multipath interference (MPI) resulting from indirect illumination significantly degrades the captured depth. Most previous works have tried to solve this problem by means of complex hardware modifications or costly computations. In this work we avoid these approaches, and propose a new technique that corrects errors in depth caused by MPI that requires no camera modifications, and corrects depth in just 10 milliseconds per frame. By observing that most MPI information can be expressed as a function of the captured depth, we pose MPI removal as a convolutional approach, and model it using a convolutional neural network. In particular, given that the input and output data present similar structure, we base our network in an autoencoder, which we train in two stages: first, we use the encoder (convolution filters) to learn a suitable basis to represent corrupted range images; then, we train the decoder (deconvolution filters) to correct depth from the learned basis from synthetically generated scenes. This approach allows us to tackle the lack of reference data, by using a large-scale captured training set with corrupted depth to train the encoder, and a smaller synthetic training set with ground truth depth to train the corrector stage of the network, which we generate by using a physically-based, time-resolved rendering. We demonstrate and validate our method on both synthetic and real complex scenarios, using an off-the-shelf ToF camera, and with only the captured incorrect depth as input.

Multi-Camera Interference Cancellation of Time-Of-Flight (tof) Cameras

DoCam: Depth Sensing with an Optical Image Stabilization Supported RGB Camera.

Iterative Error Removal for Time-of-Flight Depth Imaging

K-nearest Neighborhood Based Integration of Time-of-flight Cameras and Passive Stereo for High-Accuracy Depth Maps.

Depth Errors Analysis and Correction for Time-of-Flight (tof) Cameras.

Computational imaging with multi-camera time-of-flight systems

Data Processing Method of Time-of-flight 3D Imaging Camera

K-nearestneighborhood Basedintegrationoftime-of-flight Camerasandpassivestereoforhigh-Accuracydepthmaps ∗

Probabilistic Modeling of Multicamera Interference for Time-of-Flight Sensors

Real-Time Artifact Compensation for Depth Images of Multi-Frequency ToF

Smart Time-Multiplexing of Quads Solves the Multicamera Interference Problem

DeepToF: Off-the-Shelf Real-Time Correction of Multipath Interference in Time-of-Flight Imaging

Reliable Fusion of ToF and Stereo Data Based on Joint Depth Filter

TOF Depth Map Super-resolution Using Compressive Sensing

Accurate 3D reconstruction using a multi-phase ToF camera

A Learning Method To Optimize Depth Accuracy And Frame Rate For Time Of Flight Camera

Simultaneous Multifrequency Demodulation for Single-Shot Multiple-Path ToF Imaging

Restoration of Motion Blur in Time-of-Flight Depth Image Using Data Alignment.

High quality depth maps from stereo matching and ToF camera.

An Overview of Depth Cameras and Range Scanners Based on Time-of-Flight Technologies

SigCan: Toward Reliable ToF Estimation Leveraging Multipath Signal Cancellation on Commodity WiFi Devices