Time-of-flight based imaging in strong scattering underwater environments

Xin Jin,Dongyu Du,Jiawei Jin,Yihui Fan

DOI: https://doi.org/10.1364/oe.534170

IF: 3.8

2024-10-03

Optics Express

Abstract:Xin Jin, Dongyu Du, Jiawei Jin, Yihui Fan Time-of-flight (TOF) based underwater imaging is of great importance in practical applications due to its high image quality. Existing ... [Opt. Express 32, 37247-37259 (2024)]

optics

What problem does this paper attempt to address?

Time-of-Flight (ToF) Cameras for Underwater Situational Awareness

Kevin V. Mack,William D. Jemison,Luke K. Rumbaugh,David W. Illig,Mahesh K. Banavar

DOI: https://doi.org/10.23919/oceans40490.2019.8962695

2019-10-01

Abstract:Recently developed commercial Time-of-Flight (ToF) cameras have been used to accurately and reliably measure scene depth with high resolution in applications such as automotive LiDAR. There is a desire to adapt this technology for applications in underwater environments. In this work, we establish a methodology for using modified commercial ToF cameras in turbid water. We express the need for hardware and software modifications to the camera and demonstrate initial results in the efficacy of the camera in an underwater test scenario. We include ToF camera imagery taken under a variety of water conditions to understand the performance limitations of this technology as a function of water clarity. Target detection results from preliminary laboratory test tank experiments are presented for two different classifiers, each of which achieves high accuracy for a certain range of water conditions.
Ultra-deep three-dimensional optical coherence tomography in turbid media

Zhihua Ding,Lu Yang,Tao Han,Yun Tang,Changyong Chen,Cui Ma,Zhiyi Liu

DOI: https://doi.org/10.21203/rs.3.rs-4231224/v1

2024-01-01

Abstract:Abstract Ultra-deep three-dimensional (3D) optical coherence tomography (OCT) in turbid media is reported. Parallel detection along depth combined with scanning detection across lateral field is exploited in a swept-source OCT system to obtain the depth-resolved reflection matrices. Scanning detection enables the use of a highspeed swept source and a matched detector providing the required depth range for ultra-deep imaging. Disturbances intrinsic to scanning detection are compensated with negligible intra-frame phase variations to ensure satisfactory reconstruction by the inter-frame phase insensitive algorithm. 3D OCT imaging with an ultra-deep imaging depth of 11.5 times the scattering mean-free-path and a triple extension of the imaging depth achieved by a conventional SS-OCT is realized. The measurement time for 3D imaging (125μm×125μm×4mm ) is ~5 min, a significant leap over the time-domain matrix OCT where 6 min is taken for only one specified depth.
Time-reversal detection of multidimensional signals in underwater acoustics

Chunxiao Li,Wen Xu,Jian Li,Xianyi Gong

DOI: https://doi.org/10.1109/JOE.2010.2097830

IF: 3.883

2011-01-01

IEEE Journal of Oceanic Engineering

Abstract:Time-reversal processing (TRP) has been intensively studied for underwater object detection, however, most of the current research focuses on demonstration of the concept and development of the methods. This paper concerns the application of TRP to improving detection performance of multidimensional signals from the statistical signal processing perspective. Two commonly used time-reversal (TR) me...
Non-line-of-sight transient imaging via using time-of-flight camera

月刘,国栋李,一菲韩,钰琳崔,福郑,希浩陈,志斌孙

DOI: https://doi.org/10.1117/12.2586918

2021-03-12

Abstract:The image reconstruction of objects that are hidden from a camera’s view is a challenging issue of imaging in many fields of research due to its many potential applications in robotic vision , remote sensing and autonomous vehicles and so on. However, most of non-line-of -sight (NLOS) imaging experiments were demonstrated through spatially scanning a visible target surface with a pulsed laser and a time resolved detector, where both the system and image reconstruction are complicated. Here, we performed a transient non-line-of-sight imaging experiment by using a time-of-flight (TOF) camera, in which the images can be recovered via just capturing the light transport information encoded in multiply scattered light in a short time of 8.3 ms. It also has some advantages of lower 8W power requirements, lower cost, and shorter reconstruction time compared to the previous systems. Additionally, we propose a method of data processing based on Gaussian fitting to reduce the noises of images, where image quality can significantly improved.
Computational imaging with multi-camera time-of-flight systems

Shikhar Shrestha,Felix Heide,Wolfgang Heidrich,Gordon Wetzstein

DOI: https://doi.org/10.1145/2897824.2925928

IF: 6.2

2016-07-11

ACM Transactions on Graphics

Abstract:Depth cameras are a ubiquitous technology used in a wide range of applications, including robotic and machine vision, human-computer interaction, autonomous vehicles as well as augmented and virtual reality. In this paper, we explore the design and applications of phased multi-camera time-of-flight (ToF) systems. We develop a reproducible hardware system that allows for the exposure times and waveforms of up to three cameras to be synchronized. Using this system, we analyze waveform interference between multiple light sources in ToF applications and propose simple solutions to this problem. Building on the concept of orthogonal frequency design, we demonstrate state-of-the-art results for instantaneous radial velocity capture via Doppler time-of-flight imaging and we explore new directions for optically probing global illumination, for example by de-scattering dynamic scenes and by non-line-of-sight motion detection via frequency gating.

computer science, software engineering
Towards Real-Time Through-Obstacle Imaging Based on Compressed Sensing for Sparse Objects

Tianyi Zhou,Fazhong Shen,Qingyang Meng,Huan Li,Kuiwen Xu,Dexin Ye,Jiangtao Huangfu,Shiwei Dong,Tayeb Ahmed Denidni,Lixin Ran

DOI: https://doi.org/10.1049/iet-map.2019.0238

2019-01-01

Abstract:Although significant progress has been made in microwave imaging, real-time imaging, especially for objects behind walls or closed obstacles, remains a technical challenge. In this work, highly efficient imaging for complex-structured objects surrounded by a closed obstacle was experimentally demonstrated. The imaging equations are derived based on a combination of the inverse-scattering problem and the concept of compressed sensing. Making use of the spatial sparsity of objects and obstacles, the compressed imaging can be implemented using a time-division multi-antenna setup with reduced transmitting antennas. Owing to the spatial compressed sensing applied to the sparse imaging region and objects, the imaging time can be reduced by two orders of magnitude compared with the conventional twofold subspace-based optimisation method with a comparable imaging quality. Taking advantage of the sparsity of the entire imaging area, objects with larger relative permittivity can also be reconstructed. The proposed method can be potentially used in applications such as security examination through boxes. It also provides a new clue for solving the practicability difficulty faced by existing microwave imaging systems.
Occluded Imaging with Time-of-Flight Sensors

Achuta Kadambi,Hang Zhao,Boxin Shi,Ramesh Raskar

DOI: https://doi.org/10.1145/2836164

IF: 6.2

2016-01-01

ACM Transactions on Graphics

Abstract:We explore the question of whether phase-based time-of-flight (TOF) range cameras can be used for looking around corners and through scattering diffusers. By connecting TOF measurements with theory from array signal processing, we conclude that performance depends on two primary factors: camera modulation frequency and the width of the specular lobe (“shininess”) of the wall. For purely Lambertian walls, commodity TOF sensors achieve resolution on the order of meters between targets. For seemingly diffuse walls, such as posterboard, the resolution is drastically reduced, to the order of 10cm. In particular, we find that the relationship between reflectance and resolution is nonlinear—a slight amount of shininess can lead to a dramatic improvement in resolution. Since many realistic scenes exhibit a slight amount of shininess, we believe that off-the-shelf TOF cameras can look around corners.
High-speed object detection with a single-photon time-of-flight image sensor

Germán Mora-Martín,Alex Turpin,Alice Ruget,Abderrahim Halimi,Robert Henderson,Jonathan Leach,Istvan Gyongy

DOI: https://doi.org/10.1364/OE.435619

IF: 3.8

2021-09-30

Optics Express

Abstract:Germán Mora-Martín, Alex Turpin, Alice Ruget, Abderrahim Halimi, Robert Henderson, Jonathan Leach, Istvan Gyongy 3D time-of-flight (ToF) imaging is used in a variety of applications such as augmented reality (AR), computer interfaces, robotics and ... [Opt. Express 29, 33184-33196 (2021)]

optics
All-photon Polarimetric Time-of-Flight Imaging

Seung-Hwan Baek,Felix Heide

DOI: https://doi.org/10.48550/arXiv.2112.09278

2021-12-17

Abstract:Time-of-flight (ToF) sensors provide an imaging modality fueling diverse applications, including LiDAR in autonomous driving, robotics, and augmented reality. Conventional ToF imaging methods estimate the depth by sending pulses of light into a scene and measuring the ToF of the first-arriving photons directly reflected from a scene surface without any temporal delay. As such, all photons following this first response are typically considered as unwanted noise. In this paper, we depart from the principle of using first-arriving photons and propose an all-photon ToF imaging method by incorporating the temporal-polarimetric analysis of first- and late-arriving photons, which possess rich scene information about its geometry and material. To this end, we propose a novel temporal-polarimetric reflectance model, an efficient capture method, and a reconstruction method that exploits the temporal-polarimetric changes of light reflected by the surface and sub-surface reflection. The proposed all-photon polarimetric ToF imaging method allows for acquiring depth, surface normals, and material parameters of a scene by utilizing all photons captured by the system, whereas conventional ToF imaging only obtains coarse depth from the first-arriving photons. We validate our method in simulation and experimentally with a prototype.

Computer Vision and Pattern Recognition
Time-gated imaging through dense fog via physics-driven Swin transformer

Shaohui Jin,Ziqin Xu,Mingliang Xu,Hao Liu

DOI: https://doi.org/10.1364/oe.519662

IF: 3.8

2024-05-08

Optics Express

Abstract:Shaohui Jin, Ziqin Xu, Mingliang Xu, Hao Liu Imaging through the fog is valuable for many areas, such as autonomous driving and cosmic exploration. However, due to the influence of ... [Opt. Express 32, 18812-18830 (2024)]

optics
An Overview of Depth Cameras and Range Scanners Based on Time-of-Flight Technologies

Radu Horaud,Miles Hansard,Georgios Evangelidis,Clement Menier

DOI: https://doi.org/10.48550/arXiv.2012.06772

2020-12-12

Computer Vision and Pattern Recognition

Abstract:Time-of-flight (TOF) cameras are sensors that can measure the depths of scene-points, by illuminating the scene with a controlled laser or LED source, and then analyzing the reflected light. In this paper, we will first describe the underlying measurement principles of time-of-flight cameras, including: (i) pulsed-light cameras, which measure directly the time taken for a light pulse to travel from the device to the object and back again, and (ii) continuous-wave modulated-light cameras, which measure the phase difference between the emitted and received signals, and hence obtain the travel time indirectly. We review the main existing designs, including prototypes as well as commercially available devices. We also review the relevant camera calibration principles, and how they are applied to TOF devices. Finally, we discuss the benefits and challenges of combined TOF and color camera systems.
High Resolution, Deep Imaging Using Confocal Time-of-flight Diffuse Optical Tomography

Yongyi Zhao,Ankit Raghuram,Hyun K. Kim,Andreas H. Hielscher,Jacob T. Robinson,Ashok Veeraraghavan

DOI: https://doi.org/10.1109/TPAMI.2021.3075366

2021-05-28

Abstract:Light scattering by tissue severely limits how deep beneath the surface one can image, and the spatial resolution one can obtain from these images. Diffuse optical tomography (DOT) is one of the most powerful techniques for imaging deep within tissue -- well beyond the conventional $\sim$10-15 mean scattering lengths tolerated by ballistic imaging techniques such as confocal and two-photon microscopy. Unfortunately, existing DOT systems are limited, achieving only centimeter-scale resolution. Furthermore, they suffer from slow acquisition times and slow reconstruction speeds making real-time imaging infeasible. We show that time-of-flight diffuse optical tomography (ToF-DOT) and its confocal variant (CToF-DOT), by exploiting the photon travel time information, allow us to achieve millimeter spatial resolution in the highly scattered diffusion regime ($> 50 $ mean free paths). In addition, we demonstrate two additional innovations: focusing on confocal measurements, and multiplexing the illumination sources allow us to significantly reduce the measurement acquisition time. Finally, we rely on a novel convolutional approximation that allows us to develop a fast reconstruction algorithm, achieving a 100$\times$ speedup in reconstruction time compared to traditional DOT reconstruction techniques. Together, we believe that these technical advances serve as the first step towards real-time, millimeter resolution, deep tissue imaging using DOT.

Image and Video Processing,Optics
Enhancing the spatial resolution of time-of-flight based non-line-of-sight imaging via instrument response function deconvolution

DingJie Wang,Wei Hao,YuYuan Tian,WeiHao Xu,Yuan Tian,HaiHao Cheng,SongMao Chen,Ning Zhang,WenHua Zhu,XiuQin Su

DOI: https://doi.org/10.1364/oe.518767

IF: 3.8

2024-03-21

Optics Express

Abstract:DingJie Wang, Wei Hao, YuYuan Tian, WeiHao Xu, Yuan Tian, HaiHao Cheng, SongMao Chen, Ning Zhang, WenHua Zhu, XiuQin Su Non-line-of-sight (NLOS) imaging retrieves the hidden scenes by utilizing the signals indirectly reflected by the relay wall. Benefiting ... [Opt. Express 32, 12303-12317 (2024)]

optics
Model Study of Transient Imaging With Multi-Frequency Time-of-Flight Sensors

Hongman Wang,Hui Qiao,Jingyu Lin,Rihui Wu,Yebin Liu,Qionghai Dai

DOI: https://doi.org/10.1109/TPAMI.2020.2981574

IF: 23.6

2021-01-01

IEEE Transactions on Pattern Analysis and Machine Intelligence

Abstract:As an emerging imaging modality, transient imaging that records the transient information of light transport has significantly shaped our understanding of scenes. In spite of the great progress made in computer vision and optical imaging fields, commonly used multi-frequency time-of-flight (ToF) sensors are still afflicted with the band-limited modulation frequency and long acquisition process. To...
Simultaneous Multifrequency Demodulation for Single-Shot Multiple-Path ToF Imaging

Peyman Fayyaz Shahandashti,Paula López,Víctor Manuel Brea,Daniel García-Lesta,Miguel Heredia Conde

DOI: https://doi.org/10.1109/tci.2023.3348758

IF: 5.4

2024-02-03

IEEE Transactions on Computational Imaging

Abstract:Indirect Time-of-Flight (iToF) sensors measure the received signal's phase shift or time delay to calculate depth. In realistic conditions, however, recovering depth is challenging as reflections from secondary scattering areas or translucent objects may interfere with the direct reflection, resulting in inaccurate 3D estimates. We propose a new measurement concept including a single-shot on-chip multifrequency demodulation method with periodically-repeated ultrashort-pulsed illumination using a novel pixel array architecture to address a main limitation of conventional iToF, the Multi-Path Interference (MPI). Due to the careful hardware/software codesign, the proposed single-shot architecture provides close-to-optimal Fourier measurements to a spectral estimation algorithm that retrieves the unknown parameters of the interfering return paths in a closed form. Electrical simulations of the on-chip multifrequency demodulation circuit demonstrate the feasibility of distance retrieval in double and triple bounce conditions in a single shot with high accuracy. Furthermore, we propose a set of methods for processing the resulting sensor measurements that exploit valuable a priori information and structural constraints of the data and observe that they yield a substantial increase in accuracy.

engineering, electrical & electronic,imaging science & photographic technology
Non-Confocal 3D Reconstruction in Volumetric Scattering Scenario.

Dongyu Du,Xin Jin,Rujia Deng

DOI: https://doi.org/10.1109/tci.2023.3295599

IF: 5.4

2023-01-01

IEEE Transactions on Computational Imaging

Abstract:Real-time scattering imaging technology is of great importance for transportation in extreme weather, scientific research in the deep ocean, emergency rescue in heavy smoke, and so on. Among existing volumetric scattering imaging methods, time-of-flight (ToF) methods based on confocal imaging architecture, isolating or modeling scattering photons, present the best visual reconstruction ability. However, these methods rely on the long acquisition time to capture more effective photons and thus fail to deal with real-time imaging tasks. In this article, aiming at providing both high-speed and high-quality scattering reconstruction, a dual optical coupling scattering transmission model is proposed to accurately describe the spatial and temporal propagation of scattered photons in the non-confocal imaging architecture. Then, a non-confocal boundary migration model (NBMM) is designed to establish the mapping between scattered measurement and object information. Besides, using the special characteristic of the temporal transfer matrix, a depth reconstruction method based on re-focusing is developed to recover the 3D structure of the object. Finally, a non-confocal imaging system is built to capture photons for all pixels simultaneously and verify the effectiveness of the proposed method. The experimental results show that the proposed method can recover 3D objects located at a one-way scattering length of 22.2 transport mean free paths (TMFPs), corresponding to the round-trip scattering length of 44.4 TMFPs, which is 6.9 times more than typical non-confocal methods. It operates 600 times faster than confocal methods and only requires 100 ms exposure time, which is very beneficial to a variety of real-time scattering applications.
Scan-free Time-of-flight-based Three-Dimensional Imaging Through a Scattering Layer.

Rujia Deng,Xin Jin,Dongyu Du,Zhiheng Li

DOI: https://doi.org/10.1364/oe.492864

IF: 3.8

2023-01-01

Optics Express

Abstract:Reconstructing an object’s three-dimensional shape behind a scattering layer with a single exposure is of great significance in real-life applications. However, due to the little information captured by a single exposure while strongly perturbed by the scattering layer and encoded by free-space propagation, existing methods cannot achieve scan-free three-dimensional reconstruction through the scattering layer in macroscopic scenarios using a short acquisition time of seconds. In this paper, we proposed a scan-free time-of-flight-based three-dimensional reconstruction method based on explicitly modeling and inverting the time-of-flight-based scattering light propagation in a non-confocal imaging system. The non-confocal time-of-flight-based scattering imaging model is developed to map the three-dimensional object shape information to the time-resolved measurements, by encoding the three-dimensional object shape into the free-space propagation result and then convolving with the scattering blur kernel derived from the diffusion equation. To solve the inverse problem, a three-dimensional shape reconstruction algorithm consisting of the deconvolution and diffractive wave propagation is developed to invert the effects caused by the scattering diffusion and the free-space propagation, which reshapes the temporal and spatial distribution of scattered signal photons and recovers the object shape information. Experiments on a real scattering imaging system are conducted to demonstrate the effectiveness of the proposed method. The single exposure used in the experiment only takes 3.5 s, which is more than 200 times faster than confocal scanning methods. Experimental results show that the proposed method outperforms existing methods in terms of three-dimensional reconstruction accuracy and imaging limit subjectively and objectively. Even though the signal photons captured by a single exposure are too highly scattered and attenuated to present any valid information in time gating, the proposed method can reconstruct three-dimensional objects located behind the scattering layer of 9.6 transport mean free paths (TMFPs), corresponding to the round-trip scattering length of 19.2 TMFPs.
Fourier Analysis on Transient Imaging with a Multifrequency Time-of-Flight Camera

Jingyu Lin,Yebin Liu,Matthias B. Hullin,Qionghai Dai

DOI: https://doi.org/10.1109/CVPR.2014.419

2014-01-01

Abstract:A transient image is the optical impulse response of a scene which visualizes light propagation during an ultra-short time interval. In this paper we discover that the data captured by a multifrequency time-of-flight (ToF) camera is the Fourier transform of a transient image, and identify the sources of systematic error. Based on the discovery we propose a novel framework of frequency-domain transient imaging, as well as algorithms to remove systematic error. The whole process of our approach is of much lower computational cost, especially lower memory usage, than Heide et al.'s approach using the same device. We evaluate our approach on both synthetic and real-datasets.
Transient Imaging with a Time-of-flight Camera and Its Applications

Jing-yu Lin,Ri-hui Wu,Hong-man Wang,Ye-bin Liu

DOI: https://doi.org/10.1631/fitee.1700556

2017-01-01

Abstract:Transient imaging is a technique in photography that records the process of light propagation before it reaches a stationary state such that events at the light speed level can be observed. In this review we introduce three main models for transient imaging with a time-of-flight (ToF) camera: correlation model, frequency-domain model, and compressive sensing model. Transient imaging applications usually involve resolving the problem of light transport and separating the light rays arriving along different paths. We discuss two of the applications: imaging objects inside scattering media and recovering both the shape and texture of an object around a corner.
Simultaneous Estimation of Object Region and Depth in Participating Media Using a ToF Camera

Yuki FUJIMURA,Motoharu SONOGASHIRA,Masaaki IIYAMA

DOI: https://doi.org/10.1587/transinf.2019edp7219

2020-03-01

IEICE Transactions on Information and Systems

Abstract:Three-dimensional (3D) reconstruction and scene depth estimation from 2-dimensional (2D) images are major tasks in computer vision. However, using conventional 3D reconstruction techniques gets challenging in participating media such as murky water, fog, or smoke. We have developed a method that uses a continuous-wave time-of-flight (ToF) camera to estimate an object region and depth in participating media simultaneously. The scattered light observed by the camera is saturated, so it does not depend on the scene depth. In addition, received signals bouncing off distant points are negligible due to light attenuation, and thus the observation of such a point contains only a scattering component. These phenomena enable us to estimate the scattering component in an object region from a background that only contains the scattering component. The problem is formulated as robust estimation where the object region is regarded as outliers, and it enables the simultaneous estimation of an object region and depth on the basis of an iteratively reweighted least squares (IRLS) optimization scheme. We demonstrate the effectiveness of the proposed method using captured images from a ToF camera in real foggy scenes and evaluate the applicability with synthesized data.

computer science, information systems, software engineering

Time-of-flight based imaging in strong scattering underwater environments

Time-of-Flight (ToF) Cameras for Underwater Situational Awareness

Ultra-deep three-dimensional optical coherence tomography in turbid media

Time-reversal detection of multidimensional signals in underwater acoustics

Non-line-of-sight transient imaging via using time-of-flight camera

Computational imaging with multi-camera time-of-flight systems

Towards Real-Time Through-Obstacle Imaging Based on Compressed Sensing for Sparse Objects

Occluded Imaging with Time-of-Flight Sensors

High-speed object detection with a single-photon time-of-flight image sensor

All-photon Polarimetric Time-of-Flight Imaging

Time-gated imaging through dense fog via physics-driven Swin transformer

An Overview of Depth Cameras and Range Scanners Based on Time-of-Flight Technologies

High Resolution, Deep Imaging Using Confocal Time-of-flight Diffuse Optical Tomography

Enhancing the spatial resolution of time-of-flight based non-line-of-sight imaging via instrument response function deconvolution

Model Study of Transient Imaging With Multi-Frequency Time-of-Flight Sensors

Simultaneous Multifrequency Demodulation for Single-Shot Multiple-Path ToF Imaging

Non-Confocal 3D Reconstruction in Volumetric Scattering Scenario.

Scan-free Time-of-flight-based Three-Dimensional Imaging Through a Scattering Layer.

Fourier Analysis on Transient Imaging with a Multifrequency Time-of-Flight Camera

Transient Imaging with a Time-of-flight Camera and Its Applications

Simultaneous Estimation of Object Region and Depth in Participating Media Using a ToF Camera