Abstract:Time-of-flight (ToF) imaging has become a widespread technique for depth estimation, allowing affordable off-the-shelf cameras to provide depth maps in real time. However, multipath interference (MPI) resulting from indirect illumination significantly degrades the captured depth. Most previous works have tried to solve this problem by means of complex hardware modifications or costly computations. In this work we avoid these approaches, and propose a new technique that corrects errors in depth caused by MPI that requires no camera modifications, and corrects depth in just 10 milliseconds per frame. By observing that most MPI information can be expressed as a function of the captured depth, we pose MPI removal as a convolutional approach, and model it using a convolutional neural network. In particular, given that the input and output data present similar structure, we base our network in an autoencoder, which we train in two stages: first, we use the encoder (convolution filters) to learn a suitable basis to represent corrupted range images; then, we train the decoder (deconvolution filters) to correct depth from the learned basis from synthetically generated scenes. This approach allows us to tackle the lack of reference data, by using a large-scale captured training set with corrupted depth to train the encoder, and a smaller synthetic training set with ground truth depth to train the corrector stage of the network, which we generate by using a physically-based, time-resolved rendering. We demonstrate and validate our method on both synthetic and real complex scenarios, using an off-the-shelf ToF camera, and with only the captured incorrect depth as input.

Fisher Information Guidance for Learned Time-of-Flight Imaging

Reliable Fusion of ToF and Stereo Data Based on Joint Depth Filter

Iterative Error Removal for Time-of-Flight Depth Imaging

A Learning Method To Optimize Depth Accuracy And Frame Rate For Time Of Flight Camera

K-nearest Neighborhood Based Integration of Time-of-flight Cameras and Passive Stereo for High-Accuracy Depth Maps.

Simultaneous Estimation of Object Region and Depth in Participating Media Using a ToF Camera

Very Power Efficient Neural Time-of-Flight

DELTAR: Depth Estimation from a Light-Weight ToF Sensor and RGB Image

K-nearestneighborhood Basedintegrationoftime-of-flight Camerasandpassivestereoforhigh-Accuracydepthmaps ∗

Itof-Flow-based High Frame Rate Depth Imaging

Simultaneous Multifrequency Demodulation for Single-Shot Multiple-Path ToF Imaging

DeepToF: Off-the-Shelf Real-Time Correction of Multipath Interference in Time-of-Flight Imaging

All-photon Polarimetric Time-of-Flight Imaging

Depth Errors Analysis and Correction for Time-of-Flight (tof) Cameras.

Defogging Kinect: Simultaneous Estimation of Object Region and Depth in Foggy Scenes

Learning Inter- and Intra-frame Representations for Non-Lambertian Photometric Stereo

Deep Learning for Transient Image Reconstruction from ToF Data

Multi-Modal Neural Radiance Field for Monocular Dense SLAM with a Light-Weight ToF Sensor

Accurate 3D reconstruction using a multi-phase ToF camera

Deep End-to-End Alignment and Refinement for Time-of-Flight RGB-D Module

Deep Learning for Generating Time-of-Flight Camera Artifacts