Efficient non-line-of-sight tracking with computational neuromorphic imaging

Shuo Zhu,Zhou Ge,Chutian Wang,Jing Han,Edmund Y. Lam

DOI: https://doi.org/10.1364/ol.530066

IF: 3.6

2024-06-19

Optics Letters

Abstract:Shuo Zhu, Zhou Ge, Chutian Wang, Jing Han, Edmund Y. Lam Non-line-of-sight (NLOS) sensing is an emerging technique that is capable of detecting objects hidden behind a wall, around corners, or ... [Opt. Lett. 49, 3584-3587 (2024)]

optics

What problem does this paper attempt to address?

Non-Line-of-Sight Tracking and Mapping with an Active Corner Camera

Sheila Seidel,Hoover Rueda-Chacon,Iris Cusini,Federica Villa,Franco Zappa,Christopher Yu,Vivek K Goyal

DOI: https://doi.org/10.1038/s41467-023-39327-2

2022-08-03

Abstract:The ability to form non-line-of-sight (NLOS) images of changing scenes could be transformative in a variety of fields, including search and rescue, autonomous vehicle navigation, and reconnaissance. Most existing active NLOS methods illuminate the hidden scene using a pulsed laser directed at a relay surface and collect time-resolved measurements of returning light. The prevailing approaches include raster scanning of a rectangular grid on a vertical wall opposite the volume of interest to generate a collection of confocal measurements. These are inherently limited by the need for laser scanning. Methods that avoid laser scanning track the moving parts of the hidden scene as one or two point targets. In this work, based on more complete optical response modeling yet still without multiple illumination positions, we demonstrate accurate reconstructions of objects in motion and a 'map' of the stationary scenery behind them. The ability to count, localize, and characterize the sizes of hidden objects in motion, combined with mapping of the stationary hidden scene, could greatly improve indoor situational awareness in a variety of applications.

Image and Video Processing,Computer Vision and Pattern Recognition,Optics
Non-line-of-sight imaging and tracking of moving objects based on deep learning

JinHui He,ShuKong Wu,Ran Wei,YuNing Zhang

DOI: https://doi.org/10.1364/OE.455803

2022-05-09

Abstract:Detection of objects outside the line of sight remains a challenge in many practical applications. There have been various researches realizing 2D or 3D imaging of static hidden objects, whose aim are to improve the resolution of reconstructed images. While when it comes to the tracking of continuously moving objects, the speed of imaging and the accuracy of positioning becomes the priorities to optimize. Previous works have achieved centimeter-level or even higher precision of positioning through marking coordinates in intervals of 3 seconds to tens of milliseconds. Here a deep learning framework is proposed to realize the imaging and dynamic tracking of targets simultaneously using a standard RGB camera. Through simulation experiments, we firstly use the designed neural network to achieve positioning of a 3D mannequin with sub-centimeter accuracy (relative error under 1.8%), costing only 3 milliseconds per estimation in average. Furthermore, we apply the system to a physical scene to successfully recover the video signal of the moving target, intuitively revealing its trajectory. We demonstrate an efficient and inexpensive approach that can present the movement of objects around the corner in real time, profiting from the imaging of the NLOS scene, it is also possible to identify the hidden target. This technique can be ultilized to security surveillance, military reconnaissance, autonomous driving and other fields.
Neural Network Based 3D Tracking with a Graphene Transparent Focal Stack Imaging System

Dehui Zhang,Zhen Xu,Zhengyu Huang,Audrey Rose Gutierrez,Cameron J. Blocker,Che-Hung Liu,Miao-Bin Lien,Gong Cheng,Zhe Liu,Il Yong Chun,Jeffrey A. Fessler,Zhaohui Zhong,Theodore B. Norris

DOI: https://doi.org/10.1038/s41467-021-22696-x

IF: 16.6

2021-01-01

Nature Communications

Abstract:Recent years have seen the rapid growth of new approaches to optical imaging, with an emphasis on extracting three-dimensional (3D) information from what is normally a two-dimensional (2D) image capture. Perhaps most importantly, the rise of computational imaging enables both new physical layouts of optical components and new algorithms to be implemented. This paper concerns the convergence of two advances: the development of a transparent focal stack imaging system using graphene photodetector arrays, and the rapid expansion of the capabilities of machine learning including the development of powerful neural networks. This paper demonstrates 3D tracking of point-like objects with multilayer feedforward neural networks and the extension to tracking positions of multi-point objects. Computer simulations further demonstrate how this optical system can track extended objects in 3D, highlighting the promise of combining nanophotonic devices, new optical system designs, and machine learning for new frontiers in 3D imaging.
High-resolution non-line-of-sight imaging based on liquid crystal planar optical elements

Zhibin Zhao,Qi Zhang,Xiaoyin Li,Yinghui Guo,Mingbo Pu,Fei Zhang,Hengshuo Guo,Zewei Wang,Yulong Fan,Mingfeng Xu,Xiangang Luo

DOI: https://doi.org/10.1515/nanoph-2023-0655

IF: 7.5

2024-01-11

Nanophotonics

Abstract:Non-line-of-sight (NLOS) imaging aims at recovering hidden objects located beyond the traditional line of sight, with potential applications in areas such as security monitoring, search and rescue, and autonomous driving. Conventionally, NLOS imaging requires raster scanning of laser pulses and collecting the reflected photons from a relay wall. High-time-resolution detectors obtain the flight time of photons undergoing multiple scattering for image reconstruction. Expanding the scanning area while maintaining the sampling rate is an effective method to enhance the resolution of NLOS imaging, where an angle magnification system is commonly adopted. Compared to traditional optical components, planar optical elements such as liquid crystal, offer the advantages of high efficiency, lightweight, low cost, and ease of processing. By introducing liquid crystal with angle magnification capabilities into the NLOS imaging system, we successfully designed a large field-of-view high-resolution system for a wide scanning area and high-quality image reconstruction. Furthermore, in order to reduce the long data acquisition time, a sparse scanning method capitalizing on the correlation between measurement data to reduce the number of sampling points is thus proposed. Both the simulation and experiment results demonstrate a >20 % reduction in data acquisition time while maintaining the exact resolution.

optics,physics, applied,materials science, multidisciplinary,nanoscience & nanotechnology
Long-Range Non-Line-of-Sight Imaging Based on Projected Images from Multiple Light Fields

Xiaojie Chen,Mengyue Li,Tiantian Chen,Shuyue Zhan

DOI: https://doi.org/10.3390/photonics10010025

IF: 2.536

2022-01-01

Photonics

Abstract:Non-line-of-sight (NLOS) imaging technology has shown potential in several applications, such as intelligent driving, warfare and reconnaissance, medical diagnosis, and disaster rescue. However, most NLOS imaging systems are expensive and have a limited detection range, which hinders their utility in real-world scenarios. To address these limitations, we designed an NLOS imaging system, which is capable of long-range data acquisition. We also introduce an NLOS object imaging method based on deep learning, which makes use of long-range projected images from different light fields to reconstruct hidden objects. The method learns the mapping relationships of projected images and objects and corrects the image structure to suppress the generation of artifacts in order to improve the reconstruction quality. The results show that the proposed method produces fewer artifacts in reconstructions, which are close to human subjective perception. Furthermore, NLOS targets can be reconstructed even if the distance between the detection device and the intermediate surface exceeds 50 m.
Dynamic-excitation-based Steady-State Non-Line-of-sight Imaging Via Multi-Branch Convolutional Neural Network

Yanpeng Cao,Rui Liang,Wenbin Zhu,Bowen Zhao,Haotian Chen,Lingfeng Shen,Jiangxin Yang,Yanlong Cao,Jian Chen,Xin Li

DOI: https://doi.org/10.1016/j.optlaseng.2022.107369

IF: 5.666

2023-01-01

Optics and Lasers in Engineering

Abstract:Non-line-of-sight (NLOS) imaging technology can effectively expand the "field of view " and detect hidden targets located in "blind areas " or "dead corners " which are useful for various optical sensing applications. In this pa-per, we present a complete framework for rendering and processing dynamic-excitation-based steady-state NLOS images. First, we construct a virtual rendering pipeline for generating large-scale rendered steady-state NLOS im-ages and build a physical hardware setup for acquiring real-captured steady-state NLOS images under dynamic excitation circumstances. Second, we present an end-to-end multi-branch convolutional neural network (CNN) to reconstruct specular-reflected images of hidden objects based on the fusion of multiple dynamic-excitation -based NLOS images. Qualitative and quantitative experiments on the rendered images and real-captured images validate the effectiveness of the proposed dynamic-excitation-based steady-state NLOS method, achieving more accurate reconstruction results and exhibiting good generalization properties.
Real-time Tracking Based on Neuromrophic Vision

Hongmin Li,Pei Jing,Guoqi Li

DOI: https://doi.org/10.48550/arXiv.1510.05275

2015-10-18

Computer Vision and Pattern Recognition

Abstract:Real-time tracking is an important problem in computer vision in which most methods are based on the conventional cameras. Neuromorphic vision is a concept defined by incorporating neuromorphic vision sensors such as silicon retinas in vision processing system. With the development of the silicon technology, asynchronous event-based silicon retinas that mimic neuro-biological architectures has been developed in recent years. In this work, we combine the vision tracking algorithm of computer vision with the information encoding mechanism of event-based sensors which is inspired from the neural rate coding mechanism. The real-time tracking of single object with the advantage of high speed of 100 time bins per second is successfully realized. Our method demonstrates that the computer vision methods could be used for the neuromorphic vision processing and we can realize fast real-time tracking using neuromorphic vision sensors compare to the conventional camera.
Recent Advances on Non-Line-of-Sight Imaging: Conventional Physical Models, Deep Learning, and New Scenes

Ruixu Geng,Yang Hu,Yan Chen

DOI: https://doi.org/10.1561/116.00000019

2022-01-01

APSIPA Transactions on Signal and Information Processing

Abstract:As an emerging technology that has attracted huge attention, non-line-of-sight (NLOS) imaging can reconstruct hidden objects by analyzing the diffuse reflection on a relay surface, with broad application prospects in the fields of autonomous driving, medical imaging, and defense. Despite the challenges of low signal-to-noise ratio (SNR) and high ill-posedness, NLOS imaging has been developed rapidly in recent years. Most current NLOS imaging technologies use conventional physical models, constructing imaging models through active or passive illumination and using reconstruction algorithms to restore hidden scenes. Moreover, deep learning algorithms for NLOS imaging have also received much attention recently. This paper presents a comprehensive overview of both conventional and deep learning-based NLOS imaging techniques. Besides, we also survey new proposed NLOS scenes, and discuss the challenges and prospects of existing technologies. Such a survey can help readers have an overview of different types of NLOS imaging, thus expediting the development of seeing around corners.
PathFinder: Attention-Driven Dynamic Non-Line-of-Sight Tracking with a Mobile Robot

Shenbagaraj Kannapiran,Sreenithy Chandran,Suren Jayasuriya,Spring Berman

2024-04-08

Abstract:The study of non-line-of-sight (NLOS) imaging is growing due to its many potential applications, including rescue operations and pedestrian detection by self-driving cars. However, implementing NLOS imaging on a moving camera remains an open area of research. Existing NLOS imaging methods rely on time-resolved detectors and laser configurations that require precise optical alignment, making it difficult to deploy them in dynamic environments. This work proposes a data-driven approach to NLOS imaging, PathFinder, that can be used with a standard RGB camera mounted on a small, power-constrained mobile robot, such as an aerial drone. Our experimental pipeline is designed to accurately estimate the 2D trajectory of a person who moves in a Manhattan-world environment while remaining hidden from the camera's field-of-view. We introduce a novel approach to process a sequence of dynamic successive frames in a line-of-sight (LOS) video using an attention-based neural network that performs inference in real-time. The method also includes a preprocessing selection metric that analyzes images from a moving camera which contain multiple vertical planar surfaces, such as walls and building facades, and extracts planes that return maximum NLOS information. We validate the approach on in-the-wild scenes using a drone for video capture, thus demonstrating low-cost NLOS imaging in dynamic capture environments.

Computer Vision and Pattern Recognition,Robotics
Physics to the Rescue: Deep Non-line-of-sight Reconstruction for High-speed Imaging

Fangzhou Mu,Sicheng Mo,Jiayong Peng,Xiaochun Liu,Ji Hyun Nam,Siddeshwar Raghavan,Andreas Velten,Yin Li

DOI: https://doi.org/10.48550/arXiv.2205.01679

2022-08-06

Abstract:Computational approach to imaging around the corner, or non-line-of-sight (NLOS) imaging, is becoming a reality thanks to major advances in imaging hardware and reconstruction algorithms. A recent development towards practical NLOS imaging, Nam et al. demonstrated a high-speed non-confocal imaging system that operates at 5Hz, 100x faster than the prior art. This enormous gain in acquisition rate, however, necessitates numerous approximations in light transport, breaking many existing NLOS reconstruction methods that assume an idealized image formation model. To bridge the gap, we present a novel deep model that incorporates the complementary physics priors of wave propagation and volume rendering into a neural network for high-quality and robust NLOS reconstruction. This orchestrated design regularizes the solution space by relaxing the image formation model, resulting in a deep model that generalizes well on real captures despite being exclusively trained on synthetic data. Further, we devise a unified learning framework that enables our model to be flexibly trained using diverse supervision signals, including target intensity images or even raw NLOS transient measurements. Once trained, our model renders both intensity and depth images at inference time in a single forward pass, capable of processing more than 5 captures per second on a high-end GPU. Through extensive qualitative and quantitative experiments, we show that our method outperforms prior physics and learning based approaches on both synthetic and real measurements. We anticipate that our method along with the fast capturing system will accelerate future development of NLOS imaging for real world applications that require high-speed imaging.

Image and Video Processing,Computer Vision and Pattern Recognition
Low-latency time-of-flight non-line-of-sight imaging at 5 frames per second

Ji Hyun Nam,Eric Brandt,Sebastian Bauer,Xiaochun Liu,Marco Renna,Alberto Tosi,Eftychios Sifakis,Andreas Velten

DOI: https://doi.org/10.1038/s41467-021-26721-x

IF: 16.6

2021-11-11

Nature Communications

Abstract:Abstract Non-Line-Of-Sight (NLOS) imaging aims at recovering the 3D geometry of objects that are hidden from the direct line of sight. One major challenge with this technique is the weak available multibounce signal limiting scene size, capture speed, and reconstruction quality. To overcome this obstacle, we introduce a multipixel time-of-flight non-line-of-sight imaging method combining specifically designed Single Photon Avalanche Diode (SPAD) array detectors with a fast reconstruction algorithm that captures and reconstructs live low-latency videos of non-line-of-sight scenes with natural non-retroreflective objects. We develop a model of the signal-to-noise-ratio of non-line-of-sight imaging and use it to devise a method that reconstructs the scene such that signal-to-noise-ratio, motion blur, angular resolution, and depth resolution are all independent of scene depth suggesting that reconstruction of very large scenes may be possible.

multidisciplinary sciences
High-resolution non-line-of-sight imaging employing active focusing

Ruizhi Cao,Frederic de Goumoens,Baptiste Blochet,Jian Xu,Changhuei Yang

DOI: https://doi.org/10.1038/s41566-022-01009-8

IF: 35

2022-05-31

Nature Photonics

Abstract:Non-line-of-sight (NLOS) imaging is a rapidly developing research direction that has significant applications in autonomous vehicles, remote sensing and other areas. Existing NLOS methods primarily depend on time-gated measurements and sophisticated signal processing to extract information from scattered light. Here we introduce a method that directly manipulates light to counter the wall's scattering. This method, termed unseen non-line-of-sight casted optical aperture visibility-enhanced return (UNCOVER) focusing, operates by actively focusing light onto the hidden target using wavefront shaping. By raster scanning that focus, we can actively image the hidden object. The focus thus formed is near diffraction limited and can be substantially smaller than the object itself, thereby enabling us to perform NLOS imaging with unprecedented resolution. We demonstrate that a resolution of ~0.6 mm at a distance of 0.55 m is achievable in UNCOVER, giving us a distance-to-resolution ratio of ~970.

optics,physics, applied
An Efficient Way for Active None-Line-of-Sight: End-to-End Learned Compressed NLOS Imaging.

Chen Chang,Tao Yue,Siqi Ni,Xuemei Hu

DOI: https://doi.org/10.1007/978-981-99-8537-1_3

2024-01-01

Abstract:Non-line-of-sight imaging (NLOS) is an emerging detection technique that uses multiple reflections of a transmitted beam, capturing scenes beyond the user's field of view. Due to its high reconstruction quality, active transient NLOS imaging has been widely investigated. However, much of the existing work has focused on optimizing reconstruction algorithms but neglected the time cost during data acquisition. Conventional imaging systems use mechanical point-by-point scanning, which requires high time cost and not utilizing the sparsity of the NLOS objects. In this paper, we propose to realize NLOS in an efficient way, based upon the theory of compressive sensing (CS). To reduce data volume and acquisition time, we introduce the end-to-end CS imaging to learn an optimal CS measurement matrix for efficient NLOS imaging. Through quantitative and qualitative experimental comparison with SOTA methods, we demonstrate an improvement of at least 1.4 dB higher PSNR for the reconstructed depth map compared to using partial Hadamard sensing matrices. This work will effectively advance the real-time and practicality of NLOS.
Non-line-of-Sight Imaging via Neural Transient Fields

Siyuan Shen,Zi Wang,Ping Liu,Zhengqing Pan,Ruiqian Li,Tian Gao,Shiying Li,Jingyi Yu

DOI: https://doi.org/10.1109/tpami.2021.3076062

IF: 23.6

2021-07-01

IEEE Transactions on Pattern Analysis and Machine Intelligence

Abstract:We present a neural modeling framework for non-line-of-sight (NLOS) imaging. Previous solutions have sought to explicitly recover the 3D geometry (e.g., as point clouds) or voxel density (e.g., within a pre-defined volume) of the hidden scene. In contrast, inspired by the recent Neural Radiance Field (NeRF) approach, we use a multi-layer perceptron (MLP) to represent the neural transient field or NeTF. However, NeTF measures the transient over spherical wavefronts rather than the radiance along lines. We therefore formulate a spherical volume NeTF reconstruction pipeline, applicable to both confocal and non-confocal setups. Compared with NeRF, NeTF samples a much sparser set of viewpoints (scanning spots) and the sampling is highly uneven. We thus introduce a Monte Carlo technique to improve the robustness in the reconstruction. Experiments on synthetic and real datasets demonstrate NeTF achieves state-of-the-art performance and can provide reliable reconstructions even under semi-occlusions and on non-Lambertian materials.

computer science, artificial intelligence,engineering, electrical & electronic
Keyhole Imaging: Non-Line-of-Sight Imaging and Tracking of Moving Objects Along a Single Optical Path

Christopher A. Metzler,David B. Lindell,Gordon Wetzstein

DOI: https://doi.org/10.48550/arXiv.1912.06727

2021-01-06

Abstract:Non-line-of-sight (NLOS) imaging and tracking is an emerging technology that allows the shape or position of objects around corners or behind diffusers to be recovered from transient, time-of-flight measurements. However, existing NLOS approaches require the imaging system to scan a large area on a visible surface, where the indirect light paths of hidden objects are sampled. In many applications, such as robotic vision or autonomous driving, optical access to a large scanning area may not be available, which severely limits the practicality of existing NLOS techniques. Here, we propose a new approach, dubbed keyhole imaging, that captures a sequence of transient measurements along a single optical path, for example, through a keyhole. Assuming that the hidden object of interest moves during the acquisition time, we effectively capture a series of time-resolved projections of the object's shape from unknown viewpoints. We derive inverse methods based on expectation-maximization to recover the object's shape and location using these measurements. Then, with the help of long exposure times and retroreflective tape, we demonstrate successful experimental results with a prototype keyhole imaging system.

Computer Vision and Pattern Recognition,Signal Processing
Deep Non-line-of-sight Imaging from Under-scanning Measurements.

Yue Li,Yueyi Zhang,Juntian Ye,Feihu Xu,Zhiwei Xiong

2023-01-01

Abstract:Active confocal non-line-of-sight (NLOS) imaging has successfully enabled seeing around corners relying on high-quality transient measurements. However, acquiring spatial-dense transient measurement is time-consuming, raising the question of how to reconstruct satisfactory results from under-scanning measurements (USM). The existing solutions, involving the traditional algorithms, however, are hindered by unsatisfactory results or long computing times. To this end, we propose the first deep-learning-based approach to NLOS imaging from USM. Our proposed end-to-end network is composed of two main components: the transient recovery network (TRN) and the volume reconstruction network (VRN). Specifically, TRN takes the under-scanning measurements as input, utilizes a multiple kernel feature extraction module and a multiple feature fusion module, and outputs sufficient-scanning measurements at the high-spatial resolution. Afterwards, VRN incorporates the linear physics prior of the light-path transport model and reconstructs the hidden volume representation. Besides, we introduce regularized constraints that enhance the perception of more local details while suppressing smoothing effects. The proposed method achieves superior performance on both synthetic data and public real-world data, as demonstrated by extensive experimental results with different under-scanning grids. Moreover, the proposed method delivers impressive robustness at an extremely low scanning grid (i.e., 8$\times$8) and offers high-speed inference (i.e., 50 times faster than the existing iterative solution).
Towards Non-Line-of-Sight Photography

Jiayong Peng,Fangzhou Mu,Ji Hyun Nam,Siddeshwar Raghavan,Yin Li,Andreas Velten,Zhiwei Xiong

DOI: https://doi.org/10.48550/arxiv.2109.07783

2022-01-01

Abstract: Non-line-of-sight (NLOS) imaging is based on capturing the multi-bounce indirect reflections from the hidden objects. Active NLOS imaging systems rely on the capture of the time of flight of light through the scene, and have shown great promise for the accurate and robust reconstruction of hidden scenes without the need for specialized scene setups and prior assumptions. Despite that existing methods can reconstruct 3D geometries of the hidden scene with excellent depth resolution, accurately recovering object textures and appearance with high lateral resolution remains an challenging problem. In this work, we propose a new problem formulation, called NLOS photography, to specifically address this deficiency. Rather than performing an intermediate estimate of the 3D scene geometry, our method follows a data-driven approach and directly reconstructs 2D images of a NLOS scene that closely resemble the pictures taken with a conventional camera from the location of the relay wall. This formulation largely simplifies the challenging reconstruction problem by bypassing the explicit modeling of 3D geometry, and enables the learning of a deep model with a relatively small training dataset. The results are NLOS reconstructions of unprecedented lateral resolution and image quality.
Non-line-of-sight snapshots and background mapping with an active corner camera

Sheila Seidel,Hoover Rueda-Chacón,Iris Cusini,Federica Villa,Franco Zappa,Christopher Yu,Vivek K Goyal

DOI: https://doi.org/10.1038/s41467-023-39327-2

2023-06-21

Abstract:The ability to form reconstructions beyond line-of-sight view could be transformative in a variety of fields, including search and rescue, autonomous vehicle navigation, and reconnaissance. Most existing active non-line-of-sight (NLOS) imaging methods use data collection steps in which a pulsed laser is directed at several points on a relay surface, one at a time. The prevailing approaches include raster scanning of a rectangular grid on a vertical wall opposite the volume of interest to generate a collection of confocal measurements. These and a recent method that uses a horizontal relay surface are inherently limited by the need for laser scanning. Methods that avoid laser scanning to operate in a snapshot mode are limited to treating the hidden scene of interest as one or two point targets. In this work, based on more complete optical response modeling yet still without multiple illumination positions, we demonstrate accurate reconstructions of foreground objects while also introducing the capability of mapping the stationary scenery behind moving objects. The ability to count, localize, and characterize the sizes of hidden objects, combined with mapping of the stationary hidden scene, could greatly improve indoor situational awareness in a variety of applications.
Seeing Around Street Corners: Non-Line-of-Sight Detection and Tracking In-the-Wild Using Doppler Radar

Nicolas Scheiner,Florian Kraus,Fangyin Wei,Buu Phan,Fahim Mannan,Nils Appenrodt,Werner Ritter,Jürgen Dickmann,Klaus Dietmayer,Bernhard Sick,Felix Heide

DOI: https://doi.org/10.1109/CVPR42600.2020.00214

2020-03-31

Abstract:Conventional sensor systems record information about directly visible objects, whereas occluded scene components are considered lost in the measurement process. Non-line-of-sight (NLOS) methods try to recover such hidden objects from their indirect reflections - faint signal components, traditionally treated as measurement noise. Existing NLOS approaches struggle to record these low-signal components outside the lab, and do not scale to large-scale outdoor scenes and high-speed motion, typical in automotive scenarios. In particular, optical NLOS capture is fundamentally limited by the quartic intensity falloff of diffuse indirect reflections. In this work, we depart from visible-wavelength approaches and demonstrate detection, classification, and tracking of hidden objects in large-scale dynamic environments using Doppler radars that can be manufactured at low-cost in series production. To untangle noisy indirect and direct reflections, we learn from temporal sequences of Doppler velocity and position measurements, which we fuse in a joint NLOS detection and tracking network over time. We validate the approach on in-the-wild automotive scenes, including sequences of parked cars or house facades as relay surfaces, and demonstrate low-cost, real-time NLOS in dynamic automotive environments.

Computer Vision and Pattern Recognition,Image and Video Processing
Non-line-of-sight tracking of people at long range

Susan Chan,Ryan E. Warburton,Genevieve Gariepy,Jonathan Leach,Daniele Faccio

DOI: https://doi.org/10.1364/oe.25.010109

IF: 3.8

2017-04-24

Optics Express

Abstract:A remote-sensing system that can determine the position of hidden objects has applications in many critical real-life scenarios, such as search and rescue missions and safe autonomous driving. Previous work has shown the ability to range and image objects hidden from the direct line of sight, employing advanced optical imaging technologies aimed at small objects at short range. In this work we demonstrate a long-range tracking system based on single laser illumination and single-pixel single-photon detection. This enables us to track one or more people hidden from view at a stand-off distance of over 50 m. These results pave the way towards next generation LiDAR systems that will reconstruct not only the direct-view scene but also the main elements hidden behind walls or corners.

optics

Efficient non-line-of-sight tracking with computational neuromorphic imaging

Non-Line-of-Sight Tracking and Mapping with an Active Corner Camera

Non-line-of-sight imaging and tracking of moving objects based on deep learning

Neural Network Based 3D Tracking with a Graphene Transparent Focal Stack Imaging System

High-resolution non-line-of-sight imaging based on liquid crystal planar optical elements

Long-Range Non-Line-of-Sight Imaging Based on Projected Images from Multiple Light Fields

Dynamic-excitation-based Steady-State Non-Line-of-sight Imaging Via Multi-Branch Convolutional Neural Network

Real-time Tracking Based on Neuromrophic Vision

Recent Advances on Non-Line-of-Sight Imaging: Conventional Physical Models, Deep Learning, and New Scenes

PathFinder: Attention-Driven Dynamic Non-Line-of-Sight Tracking with a Mobile Robot

Physics to the Rescue: Deep Non-line-of-sight Reconstruction for High-speed Imaging

Low-latency time-of-flight non-line-of-sight imaging at 5 frames per second

High-resolution non-line-of-sight imaging employing active focusing

An Efficient Way for Active None-Line-of-Sight: End-to-End Learned Compressed NLOS Imaging.

Non-line-of-Sight Imaging via Neural Transient Fields

Keyhole Imaging: Non-Line-of-Sight Imaging and Tracking of Moving Objects Along a Single Optical Path

Deep Non-line-of-sight Imaging from Under-scanning Measurements.

Towards Non-Line-of-Sight Photography

Non-line-of-sight snapshots and background mapping with an active corner camera

Seeing Around Street Corners: Non-Line-of-Sight Detection and Tracking In-the-Wild Using Doppler Radar

Non-line-of-sight tracking of people at long range