Adaptive Lighting for Data-Driven Non-Line-of-Sight 3D Localization and Object Identification

Sreenithy Chandran,Suren Jayasuriya

DOI: https://doi.org/10.48550/arXiv.1905.11595

2019-07-27

Abstract:Non-line-of-sight (NLOS) imaging of objects not visible to either the camera or illumination source is a challenging task with vital applications including surveillance and robotics. Recent NLOS reconstruction advances have been achieved using time-resolved measurements which requires expensive and specialized detectors and laser sources. In contrast, we propose a data-driven approach for NLOS 3D localization and object identification requiring only a conventional camera and projector. To generalize to complex line-of-sight (LOS) scenes with non-planar surfaces and occlusions, we introduce an adaptive lighting algorithm. This algorithm, based on radiosity, identifies and illuminates scene patches in the LOS which most contribute to the NLOS light paths, and can factor in system power constraints. We achieve an average identification of 87.1% object identification for four classes of objects, and average localization of the NLOS object's centroid with a mean-squared error (MSE) of 1.97 cm in the occluded region for real data taken from a hardware prototype. These results demonstrate the advantage of combining the physics of light transport with active illumination for data-driven NLOS imaging.

Image and Video Processing,Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The problems that this paper attempts to solve are 3D positioning and object recognition in non - line - of - sight (NLOS) imaging. Specifically, the paper focuses on how to achieve precise positioning and recognition of these hidden objects when neither the camera nor the illumination source can directly view the target object. This challenge has important application value in fields such as autonomous vehicle collision avoidance, search and rescue operations, and industrial safety inspections.

To overcome the limitations of traditional NLOS reconstruction methods that require expensive and specialized time - domain measurement devices (such as time - resolved detectors and laser sources), the authors propose a data - driven method that can complete NLOS 3D positioning and object recognition tasks using only conventional cameras and projectors. In addition, in view of the non - planar surfaces and occlusion problems in complex line - of - sight (LOS) scenarios, the paper introduces an adaptive illumination algorithm. Based on the radiosity principle, this algorithm can identify and illuminate the LOS areas that contribute the most to the NLOS light path and consider the optimization of light distribution under system power constraints.

Through this method, the authors achieved an average object recognition rate of 87.1% for four types of objects and an average positioning accuracy with a mean - square error (MSE) of 1.97 cm for the center positions of NLOS objects in the occluded area. These results demonstrate the advantages of combining the physical principles of light transmission with active illumination techniques in data - driven NLOS imaging.