Abstract:3D time-of-flight (ToF) cameras have recently attracted widespread attention because of their potential in wide applications. However, the lack of versatile 3D image signal processor (ISP) is still an obstacle that hinders their popularity. The main challenges lie in the management of high-throughput raw data and the complexity of 3D image processing in hardware. To address these challenges, this paper presents a top-down 3D ISP with high real-time performance and high energy efficiency for indirect ToF (IToF) imaging. First, we propose configurable cache strategies to improve the efficiency of non-streaming memory accesses and support different IToF sensors. Then, to resolve the phase ambiguity problem commonly existing in IToF sensors, we develop a hardware-optimized cascaded multi-frequency fusion method, which not only supports up to six modulation frequencies but also facilitates the pixel-wise detection of multipath interference. Moreover, we propose a scalable calculation engine for depth calculation and calibration, 3D image enhancement, and post-processing. The pipeline is prototyped on a Xilinx ZCU102 FPGA board. Results show that the 3D ISP has negligible precision loss and low latency, achieving 53.35 x energy efficiency improvement compared with the implementation on Intel CPU and 55.45 x improvement over the ARM CPU implementation. When running at 75 MHz, the 3D ISP can support 1280 x 960 depth, intensity, and point cloud output at a frame rate of 30 f/s.

Efficient and Fast Implementation of Embedded Time-of-Flight Ranging System Based on FPGAs

FPGA Implementation of Non-Parametric Stereo Matching Algorithm

High-Speed Optical 3D Measurement Sensor for Industrial Application

FPGA implementation of full HD real-time depth estimation

Time-of-flight Ranging Method by Measuring Pulse Laser Phase Based on Field-Programmable Gate Array Digital Mixing

Research of Fast 3d Imaging Based on Multiple Mode

Development of high-speed and highly integrated CCD laser range sensor based on FPGA

High-Speed Phase Structured Light Integrated Architecture on FPGA

A Configurable and Real-Time Multi-Frequency 3D Image Signal Processor for Indirect Time-of-Flight Sensors

Full High-Definition Real-Time Depth Estimation for Three-Dimensional Video System

High Speed Image Acquisition System for 3D Reconstruction

Full Hd Real-Time Depth Estimation Algorithm and Hardware Implementation

Prototype Design of a Time-of-flight Camera Based on Xilinx Zynq7000 SoC Platform

An FPGA-based Real-Time Simultaneous Localization and Mapping System.

FPGA-based real-time object tracking for mobile robot

The Short-Range, High-Accuracy Compact Pulsed Laser Ranging System

A wide-spectrum and high-speed CMOS system based on FPGA

FPGA-Based Feature Extraction and Tracking Accelerator for Real-Time Visual SLAM

Fast Stereo 3d Imaging Based On Random Speckle Projection And Its Fpga Implementation

Optimized Design Of A Led-Array-Based Tof Range Imaging Sensor For Fast 3-D Shape Measurement

FPGA-Based High-Speed and Resource-Efficient 3D Reconstruction for Structured Light System