Depth-prior-based range-gated 3D imaging method: integration of optical property into visual learning

Jinming Gao,Xiaoquan Liu,Zhaopeng Yang

DOI: https://doi.org/10.1364/oe.531362

IF: 3.8

2024-10-16

Optics Express

Abstract:Jinming Gao, Xiaoquan Liu, Zhaopeng Yang 3D Range-gated Imaging (3DRGI) has great potential for long-range detection in adverse weather conditions. Recently, vision-guided 3DRGI ... [Opt. Express 32, 39355-39368 (2024)]

optics

What problem does this paper attempt to address?

Multi-image-distance Imaging System for Extending Depth-of-field

Jixiang Tang,Xuanyin Wang,Huan Zhou,Jiayu Ji,Zhengxiao Li,Zijian Ye

DOI: https://doi.org/10.1016/j.ijleo.2023.170965

IF: 3.1

2023-01-01

Optik

Abstract:Depth of field (DoF) defines the longitudinal range of objects in sharp focus. Imaging systems for machine vision are generally limited to a small DoF by various conditions, constraining the visual inspection of objects with large depth. In this paper, we develop a multi-image-distance imaging system, which can extend the DoF by capturing multi-focus images for fusion. A three-way beam-splitter prism is designed to support independent imaging of three image sensors. Through adjusting the sensors to different back focal lengths, multiple images focusing at different depths are captured in a single exposure. Finally, the focused regions of each image are fused for extending the DoF. Analyses of the proposed DoF extension strategy illustrate excellent enhancement more than three times. Moreover, we introduce a DoF measurement method based on the estimation of the point spread function, and measurement experiments are conducted using the fabricated prototype. Imaging experiments for various scenes demonstrate great imaging quality and DoF extension performance. The proposed multi-image-distance imaging system provides an effective and practical DoF extension solution for close-range visual inspection.
Research progress of depth detection in vision measurement: A novel project of bifocal imaging system for 3D measurement

Anhu Li,Ye Ding,Wei Wang,Yongjian Zhu,Zhizhong Li

DOI: https://doi.org/10.1117/12.2022557

2013-01-01

Abstract:The paper reviews the recent research progresses of vision measurement. The general methods of the depth detection used in the monocular stereo vision are compared with each other. As a result, a novel bifocal imaging measurement system based on the zoom method is proposed to solve the problem of the online 3D measurement. This system consists of a primary lens and a secondary one with the different focal length matching to meet the large-range and high-resolution imaging requirements without time delay and imaging errors, which has an important significance for the industry application.
Three-dimensional Active Imaging with Maximum Depth Range

Xiuda Zhang,Huimin Yan

DOI: https://doi.org/10.1364/ao.50.001682

IF: 1.9

2011-01-01

Applied Optics

Abstract:In traditional three-dimensional (3D) active imaging methods, the detection depth range is observed to increase linearly with the detection time, and the intensity information was not fully utilized. However, by encoding the relative values into pseudovalues, the intensity information was fully utilized, and we found the maximum detection depth range increases exponentially with the detection time. Furthermore, we present a 3D imaging system capable of exponentially expanding the detection depth range. A 3D scene reconstruction was undertaken with the targets placed at a distance of 600-1100m. Experimental results indicate that the method expands the detection depth range exponentially without distance resolution loss as compared with the conventional method. (C) 2011 Optical Society of America
Gain-modulated three-dimensional active imaging with depth-independent depth accuracy.

Chenfei Jin,Xiudong Sun,Yuan Zhao,Yong Zhang,Liping Liu

DOI: https://doi.org/10.1364/OL.34.003550

IF: 3.6

2009-01-01

Optics Letters

Abstract:We present a three-dimensional imaging method using a pulsed laser as a flood illuminating source and an intensified camera as the receiver with exponentially modulated gain. The depth map of a scene is obtained from two intensity images and the depth accuracy is independent of the depth of the target in the scene. We demonstrate a depth-independent depth accuracy of 0.32 m in an indoor experiment and obtain a depth map of an outdoor scene ranging from 150 to 180 m under a lower signal to noise ratio condition.
Range-Intensity-Profile-Guided Gated Light Ranging and Imaging Based on a Convolutional Neural Network

Chenhao Xia,Xinwei Wang,Liang Sun,Yue Zhang,Bo Song,Yan Zhou

DOI: https://doi.org/10.3390/s24072151

IF: 3.9

2024-03-28

Sensors

Abstract:Three-dimensional (3D) range-gated imaging can obtain high spatial resolution intensity images as well as pixel-wise depth information. Several algorithms have been developed to recover depth from gated images such as the range-intensity correlation algorithm and deep-learning-based algorithm. The traditional range-intensity correlation algorithm requires specific range-intensity profiles, which are hard to generate, while the existing deep-learning-based algorithm requires large number of real-scene training data. In this work, we propose a method of range-intensity-profile-guided gated light ranging and imaging to recover depth from gated images based on a convolutional neural network. In this method, the range-intensity profile (RIP) of a given gated light ranging and imaging system is obtained to generate synthetic training data from Grand Theft Auto V for our range-intensity ratio and semantic network (RIRS-net). The RIRS-net is mainly trained on synthetic data and fine-tuned with RIP data. The network learns both semantic depth cues and range-intensity depth cues in the synthetic data, and learns accurate range-intensity depth cues in the RIP data. In the evaluation experiments on both a real-scene and synthetic test dataset, our method shows a better result compared to other algorithms.

engineering, electrical & electronic,chemistry, analytical,instruments & instrumentation
Long-range photon-efficient 3D imaging without range ambiguity

Chen Dai,Wen-Long Ye,Chao Yu,Xin Huang,Zheng-Ping Li,Feihu Xu

DOI: https://doi.org/10.1364/ol.485127

IF: 3.6

2023-03-18

Optics Letters

Abstract:Chen Dai, Wen-Long Ye, Chao Yu, Xin Huang, Zheng-Ping Li, Feihu Xu Single-photon light detection and ranging (LiDAR) has broad applications ranging from remote sensing to target recognition. In most cases, ... [Opt. Lett. 48, 1542-1545 (2023)]

optics
Research on 3D Reconstruction by Multi-directional Range Gated Laser Imaging

Sining Li,Chao Li,Wei Lu,Dayong Zhang,Zhipeng Tu

DOI: https://doi.org/10.1109/IMCCC.2014.11

2014-01-01

Abstract:The objective of the research is to prove the 3D reconstruction capability of the range-gated laser imaging technique. To investigate this, we have designed and developed a whole set of system for high precision three-dimensional imaging. The system uses a Nd: YAG electro-optical Q-switched 532nm laser as transmitter, an image intensify tube with double micro channel plate as gated sensor, and all the components are controlled by a trigger control unit with accuracy of sub nanosecond. For imaging various aspects of the target object, a multi-directional imaging scheme is designed, experiments are performed to obtain the sequences of images. A set of algorithms is also developed and used to reconstruct the 3D image basing on centroid algorithm. Experimental results imply that the multidirectional imaging method is efficient and feasible.
An LED-array-based Range Imaging System Used for Enhancing Three-Dimensional Imaging

Huanqin Wang,Jun Xu,Deyong He,Tianpeng Zhao,Anting Wang,Hai Ming,Deyi Kong

DOI: https://doi.org/10.1117/12.869446

2010-01-01

Abstract:An LED-array-based range imaging system is proposed for three-dimensional (3-D) shape measurement. The range image is obtained by time-division electronic scanning of the LED Time-of-Flight (TOF) range finders in array, and no complex mechanical scanning is needed. By combining with a low cost CCD/CMOS sensor for capturing the twodimensional (2-D) image, the proposed range imaging system can be used to accomplish a high quality 3-D imaging. A sophisticated co-lens optical path is designed to assure the natural registration between the range image and 2-D image. Experimental tests for evaluation of the imaging system performance are described. It was found that the 3-D images can be acquired at a rate of 10 frames per second with a depth resolution better than 5mm in the range of 50 - 1000mm, which is sufficient for many practical applications, including the obstacle detection in robotics, machine automation, 3-D vision, virtual reality games and 3-D video.
Deep Learning Based Single-Photon 3D Imaging with Multiple Returns

Hao Tan,Jiayong Peng,Zhiwei Xiong,Dong Liu,Xin Huang,Zheng-Ping Li,Yu Hong,Feihu Xu

DOI: https://doi.org/10.1109/3dv50981.2020.00130

2020-01-01

Abstract:Single-photon avalanche diode (SPAD) has been widely used in active 3D imaging due to its extremely high photon sensitivity and picosecond time resolution. However, long-range active 3D imaging is still a great challenge, since only a few signal photons mixed with strong background noise can return from multiple reflectors of the scene due to the divergence of the light beam and the receiver's field of view (FoV), which would bring considerable distortion and blur to the recovered depth map. In this paper, we propose a deep learning based depth reconstruction method for long range single-photon 3D imaging where the “multiple-returns” issue exists. Specifically, we model this problem as a deblurring task and design a multi-scale convolutional neural network combined with elaborate loss functions, which promote the reconstruction of an accurate depth map with fine details and clear boundaries of objects. The proposed method achieves superior performance over several different sizes of receiver's FoV on a synthetic dataset compared with existing state-of-the-art methods and the trained model under a specific FoV has a strong generalization capability across different sizes of FoV, which is essential for practical applications. Moreover, we conduct outdoor experiments and demonstrate the effectiveness of our method in a real-world long range imaging system.
Long Range 3D Imaging Through Atmospheric Obscurants Using Array-Based Single-Photon LiDAR.

Peng-Yu Jiang,Zheng-Ping Li,Wen-Long Ye,Yu Hong,Chen Dai,Xin Huang,Shui-Qing Xi,Jie Lu,Da-Jian Cui,Yuan Cao,Feihu Xu,Jian-Wei Pan

DOI: https://doi.org/10.1364/oe.487560

IF: 3.8

2023-01-01

Optics Express

Abstract:Single-photon light detection and ranging (LiDAR) has emerged as a strong candidate technology for active imaging applications. In particular, the single-photon sensitivity and picosecond timing resolution permits high-precision three-dimensional (3D) imaging capability through atmospheric obscurants including fog, haze and smoke. Here we demonstrate an array-based single-photon LiDAR system, which is capable of performing 3D imaging in atmospheric obscurant over long ranges. By adopting the optical optimization of system and the photon-efficient imaging algorithm, we acquire depth and intensity images through dense fog equivalent to 2.74 attenuation lengths at distances of 13.4 km and 20.0 km. Furthermore, we demonstrate real-time 3D imaging for moving targets at 20 frames per second in mist weather conditions over 10.5 km. The results indicate great potential for practical applications of vehicle navigation and target recognition in challenging weather.
Single-photon Computational 3D Imaging at 45 Km

Zheng-Ping Li,Xin Huang,Yuan Cao,Bin Wang,Yu-Huai Li,Weijie Jin,Chao Yu,Jun Zhang,Qiang Zhang,Cheng-Zhi Peng,Feihu Xu,Jian-Wei Pan

DOI: https://doi.org/10.1364/prj.390091

IF: 7.6

2019-01-01

Photonics Research

Abstract:Single-photon light detection and ranging (lidar) offers single-photon sensitivity and picosecond timing resolution, which is desirable for high-precision three-dimensional (3D) imaging over long distances. Despite important progress, further extending the imaging range presents enormous challenges because only a few echo photons return and are mixed with strong noise. Here, we tackled these challenges by constructing a high-efficiency, low-noise coaxial single-photon lidar system and developing a long-range-tailored computational algorithm that provides high photon efficiency and good noise tolerance. Using this technique, we experimentally demonstrated active single-photon 3D imaging at a distance of up to 45 km in an urban environment, with a low return-signal level of ～1 photon per pixel. Our system is feasible for imaging at a few hundreds of kilometers by refining the setup, and thus represents a step towards low-power and high-resolution lidar over extra-long ranges.
A LED-array-based Range Imaging System with Time-to-Digital Converter for 3D Shape Acquisition

Huanqin Wang,Deyi Kong,Jun Xu,Deyong He

DOI: https://doi.org/10.1109/cisp.2010.5648188

2010-01-01

Abstract:A LED-array-based range imaging system is presented for capturing the depth information of three-dimensional (3D) objects. The range image is obtained by time-division electronic scanning of the LED Time-of-Flight (TOF) range finders in array, and no complex mechanical scanning is needed. Millimeter ranging resolution can be reached thanks to the use of Time-to-Digital Converter (TDC) with picosecond resolution and some special analog phased-lock loop (APLL) filter techniques. By close packaging LED die array, a high spatial resolution and large pixels version of the proposed system can be achieved easily. Experimental tests for evaluation of the system performance are adequately described. The results show the depth resolution can be better than 5 mm in the range of 300-1000 mm, and a higher resolution 0.5 mm can be achieved when the measurement time of a depth pixel is extended to 5 s, which is sufficient for many applications, including computer vision, machine automation, 3D object recognition, reverse engineering and virtual reality.
All-Time Single-Photon 3d Imaging Over 21 Km

Zheng-Ping Li,Xin Huang,Yuan Cao,Bin Wang,Yu-Huai Li,Jun Zhang,Qiang Zhang,Cheng-Zhi Peng,Feihu Xu,Jian-Wei Pan

DOI: https://doi.org/10.1364/cleo_si.2019.sm1n.1

2019-01-01

Abstract:We experimentally demonstrate active three-dimensional (3D) imaging at a range of up to 21.6 km in daylight by constructing a high-efficiency single-photon LiDAR system and developing a long-range-tailored computational algorithm. (C) 2019 The Author(s)
Improved range-gating 3D imaging laser radar

Fan Wu,Yong Zhang,Jiang He,Long Wu,Chenfei Jin,Yuan Zhao

DOI: https://doi.org/10.3969/j.issn.1007-2276.2011.12.015

2011-01-01

Abstract:Despite of being the hot-spot of laser radar research, scannerless laser radars are still in the stage of experimental research and limited in application, due to the immaturity of array detectors which are used to obtain range information. The imaging speed of existing range gated lidar is too low that restricts the application of system. This paper proposes an improved range gated 3D imaging lidar system based on integral array detector ICCD, with rapid scannerless 3D image function. The system performances match both experimentally and theoretically. Experimental results show that the system range accuracy, up to 0.1 m, coincides with theoretical analysis. The proposed system promises a sound application prospect in distinguished fields including target detection and recognition, robotic vision and fast 3D modeling of architectures.
High-resolution 3D Imaging Through Dense Camouflage Nets Using Single-Photon LiDAR

Peng-Yu Jiang,Zheng-Ping Li,Wen-Long Ye,Ziheng Qiu,Da-Jian Cui,Feihu Xu

DOI: https://doi.org/10.3788/ai.2024.10001

2024-01-01

Abstract:The single-photon sensitivity and picosecond time resolution of single-photon light detection and ranging (LiDAR) can provide a full-waveform profile for retrieving the three-dimentional (3D) profile of the target separated from foreground clutter. This capability has made single-photon LiDAR a solution for imaging through obscurant, camouflage nets, and semitransparent materials. However, the obstructive presence of the clutter and limited pixel numbers of single-photon detector arrays still pose challenges in achieving high-quality imaging. Here, we demonstrate a single-photon array LiDAR system combined with tailored computational algorithms for high-resolution 3D imaging through camouflage nets. For static targets, we develop a 3D sub-voxel scanning approach along with a photon-efficient deconvolution algorithm. Using this approach, we demonstrate 3D imaging through camouflage nets with a 3× improvement in spatial resolution and a 7.5× improvement in depth resolution compared with the inherent system resolution. For moving targets, we propose a motion compensation algorithm to mitigate the net's obstructive effects, achieving video-rate imaging of camouflaged scenes at 20 frame/s. More importantly, we demonstrate 3D imaging for complex scenes in various outdoor scenarios and evaluate the advanced features of single-photon LiDAR over a visible-light camera and a mid-wave infrared (MWIR) camera. The results point a way forward for high-resolution real-time 3D imaging of multi-depth scenarios.
Extending the depth-of-field of computational ghost imaging: Computational refocus via in situ point spread function estimation

Mengchao Ma,Wenbo Liang,Fushun Qin,Qingtian Guan,Xiang Zhong,Huaxia Deng,Ziwei Wang

DOI: https://doi.org/10.1063/5.0177211

IF: 4

2024-01-08

Applied Physics Letters

Abstract:Capturing details of objects beyond the focal plane is challenging due to the limited depth-of-field (DoF) of optical systems. Here, we report a computational refocusing ghost Imaging (CRGI) method to extend the DoF of computational ghost imaging (CGI) systems. An ultra-fast and in situ point spread function (PSF) estimation method is put forward utilizing the optical characterization of the system and compressive sensing modulation. The PSF distribution is measured with in situ compressive sensing algorithm according to reciprocity property using the same CGI system. The convolution of PSFs of various depths with modulation patterns is reshaped into measurement matrices to computationally refocus objects at different depths. From one measurement, CRGI can rebuild distinct and well-focused images of multiple objects at different depths. According to experiments, CRGI can nearly quadruple the DoF of typical CGI methods. CRGI represents a significant advancement in CGI domain by computationally surpassing the optical DoF limitations. This discovery enables recording object features beyond the focus plane using extended depth-of-field.

physics, applied
Depth Estimation of Multi-Depth Objects Based on Computational Ghost Imaging System

Wenwen Zhang,Daquan Yu,Yongcheng Han,Weiji He,Qian Chen,Ruiqing He

DOI: https://doi.org/10.1016/j.optlaseng.2021.106769

IF: 5.666

2022-01-01

Optics and Lasers in Engineering

Abstract:Computational ghost imaging (CGI) can estimate the depth of an object by evaluating the degree of defocus in the reconstructed images. However, this technology has not yet realized the depth estimation of multi-depth objects, because the defocus and in-focus phenomenon of different objects may be observed in a reconstructed image, which affects the performance of the evaluation functions. In this paper, we first analyze the formed images of multi-depth object and select the gradient domain as the image transformation space. The images are formed by the algorithm of compressed sensing based on TV norm which suppress the background noise and the effect of the defocused image. Furthermore, the deviation-based correlation (DBC) is chosen to evaluate the degree of defocus. Finally, in order to improve the efficiency, we propose the depth estimation strategy using the variable resolution speckles, which reduces the required depth slices by similar to 42% for detecting the object of interest in a given system. This research promotes the application of CGI in the field of depth-imaging.
Depth-prior-based LiDAR point cloud de-noising method leveraging range-gated imaging

Zhaopeng Yang,Xiaoquan Liu

DOI: https://doi.org/10.1364/OL.530278

2024-09-15

Abstract:Light Detection and Ranging (LiDAR) has been widely adopted in modern self-driving vehicles and mobile robotics, providing 3D information of the scene and surrounding objects. However, LiDAR systems suffer from many kinds of noise, and its noisy point clouds degrade downstream tasks. Existing LiDAR point cloud de-noising methods are time-consuming or cannot deal with the noise caused by occlusions or penetrating transparent surfaces. In this paper, we introduce a depth-prior-based LiDAR point clouds de-noising method to deal with all types of noise in LiDAR point clouds in real time. The depth prior is derived from the fundamental principles of range-gated imaging and divides the depth of field into three parts, which can provide an effective depth signal. The LiDAR point cloud, which is acquired in synchronization with gated images, is projected into a depth map, and points whose depth is inconsistent with the depth prior can be regarded as noise and can be removed, finally. We conduct an ablation study and compare the proposed method with existing de-noising methods using the Gated2Depth dataset, which is to our knowledge the first long-range-gated dataset specifically designed for 3D detection in adverse weather conditions and includes all the necessary data. The results demonstrated that the proposed method achieves superior performance across all metrics.
Depth Sensing Beyond LiDAR Range

Kai Zhang,Jiaxin Xie,Noah Snavely,Qifeng Chen

DOI: https://doi.org/10.48550/arXiv.2004.03048

2020-04-07

Abstract:Depth sensing is a critical component of autonomous driving technologies, but today's LiDAR- or stereo camera-based solutions have limited range. We seek to increase the maximum range of self-driving vehicles' depth perception modules for the sake of better safety. To that end, we propose a novel three-camera system that utilizes small field of view cameras. Our system, along with our novel algorithm for computing metric depth, does not require full pre-calibration and can output dense depth maps with practically acceptable accuracy for scenes and objects at long distances not well covered by most commercial LiDARs.

Computer Vision and Pattern Recognition
Three-dimensional ghost imaging based on differential optical path

Jie Cao,Fanghua Zhang,Kaiyu Zhang,Fei Wang,Qun Hao,Yami Fang,Qiang Luo

DOI: https://doi.org/10.1117/12.2523306

2019-01-01

Abstract:We present a novel structure based on differential optical path (DOP). The performance of three-dimensional ghost imaging (3DGI) is improved by DOP with high sensitivity and suppressed common noise because of the benefits of extracting zero-crossing point (i.e., interesting target position). Simulation results agree well with the theoretical analysis. Moreover, the relation between time slice and the signal-noise-ratio of 3DGI is discussed, and the optimal differential distance is obtained, thus motivating the development of a high-performance 3DGI.

Depth-prior-based range-gated 3D imaging method: integration of optical property into visual learning

Multi-image-distance Imaging System for Extending Depth-of-field

Research progress of depth detection in vision measurement: A novel project of bifocal imaging system for 3D measurement

Three-dimensional Active Imaging with Maximum Depth Range

Gain-modulated three-dimensional active imaging with depth-independent depth accuracy.

Range-Intensity-Profile-Guided Gated Light Ranging and Imaging Based on a Convolutional Neural Network

Long-range photon-efficient 3D imaging without range ambiguity

Research on 3D Reconstruction by Multi-directional Range Gated Laser Imaging

An LED-array-based Range Imaging System Used for Enhancing Three-Dimensional Imaging

Deep Learning Based Single-Photon 3D Imaging with Multiple Returns

Long Range 3D Imaging Through Atmospheric Obscurants Using Array-Based Single-Photon LiDAR.

Single-photon Computational 3D Imaging at 45 Km

A LED-array-based Range Imaging System with Time-to-Digital Converter for 3D Shape Acquisition

All-Time Single-Photon 3d Imaging Over 21 Km

Improved range-gating 3D imaging laser radar

High-resolution 3D Imaging Through Dense Camouflage Nets Using Single-Photon LiDAR

Extending the depth-of-field of computational ghost imaging: Computational refocus via in situ point spread function estimation

Depth Estimation of Multi-Depth Objects Based on Computational Ghost Imaging System

Depth-prior-based LiDAR point cloud de-noising method leveraging range-gated imaging

Depth Sensing Beyond LiDAR Range

Three-dimensional ghost imaging based on differential optical path