Abstract:Light detection and ranging (LiDAR) technology, a cutting-edge advancement in mobile applications, presents a myriad of compelling use cases, including enhancing low-light photography, capturing and sharing 3D images of fascinating objects, and elevating the overall augmented reality (AR) experience. However, its widespread adoption has been hindered by the prohibitive costs and substantial power consumption associated with its implementation in mobile devices. To surmount these obstacles, this paper proposes a low-power, low-cost, single-photon avalanche detector (SPAD)-based system-on-chip (SoC) which packages the microlens arrays (MLAs) and a lightweight RGB-guided sparse depth imaging completion neural network for 3D LiDAR imaging. The proposed SoC integrates an 8 × 8 SPAD macropixel array with time-to-digital converters (TDCs) and a charge pump, fabricated using a 180 nm bipolar-CMOS-DMOS (BCD) process. Initially, the primary function of this SoC was limited to serving as a ranging sensor. A random MLA-based homogenizing diffuser efficiently transforms Gaussian beams into flat-topped beams with a 45° field of view (FOV), enabling flash projection at the transmitter. To further enhance resolution and broaden application possibilities, a lightweight neural network employing RGB-guided sparse depth complementation is proposed, enabling a substantial expansion of image resolution from 8 × 8 to quarter video graphics array level (QVGA; 256 × 256). Experimental results demonstrate the effectiveness and stability of the hardware encompassing the SoC and optical system, as well as the lightweight features and accuracy of the algorithmic neural network. The state-of-the-art SoC-neural network solution offers a promising and inspiring foundation for developing consumer-level 3D imaging applications on mobile devices.

The VLSI Implementation of a High-Resolution Depth-Sensing SoC Based on Active Structured Light

Sad-Based Stereo Vision Machine on A System-On-Programmable-Chip (Sopc)

Calibration-free Deep Optics for Depth Estimation with Precise Simulation

Embedded Three-Dimensional Shape Measurement System with Microprojector

Full Hd Real-Time Depth Estimation Algorithm and Hardware Implementation

Real-time Stereo Vision System Using Adaptive Weight Cost Aggregation Approach

Design of Structured Light Depth Detection System Based on FPGA

An Efficient High-Throughput Structured-Light Depth Engine

Hardware Solution of Real-Time Depth Estimation Based on Stereo Vision

Full High-Definition Real-Time Depth Estimation for Three-Dimensional Video System

FPGA implementation of full HD real-time depth estimation

Stereo Matching Accelerator With Re-Computation Scheme and Data-Reused Pipeline for Autonomous Vehicles

A 40nm 2TOPS/W Depth-Completion Neural Network Accelerator SoC With Efficient Depth Engine for Realtime LiDAR Systems

Efficient Active Depth Sensing by Laser Speckle Projection System

A 256 × 256 LiDAR Imaging System Based on a 200 mW SPAD-Based SoC with Microlens Array and Lightweight RGB-Guided Depth Completion Neural Network

Improving RealSense by Fusing Color Stereo Vision and Infrared Stereo Vision for the Visually Impaired

ÆTIOLOGY OF JUVENILE-ONSET DIABETES A Prospective Study

Addressable Structured Light System Using Metasurface Optics and an Individually Addressable VCSEL Array

High-Speed Optical 3D Measurement Sensor for Industrial Application

Long-term efficacy of 'ready-to-drink' protein substitute in phenylketonuria.

A MEMS-based real-time structured light 3-D measuring architecture on FPGA