Abstract:Domain of stereo vision is highly important in the fields of autonomous cars, video tolling, robotics, and aerial surveys. The specific feature of this domain is that we should handle not only the pixel-by-pixel 2D processing in one image but also the 3D processing for depth estimation by comparing information about a scene from several images with different perspectives. This feature brings challenges to memory resource utilization, because an extra dimension of data has to be buffered. Due to the memory limitation, few of previous stereo vision implementations provide both accurate and high-speed processing for high-resolution images at the same time. To achieve domain-specific acceleration for stereo vision, the memory limitation has to be addressed. This article uses a Mini-Census ADaptive Support Region (MCADSR) stereo matching algorithm as a case study due to its high accuracy and representative operations in this domain. To relieve the memory limitation and achieve high-speed processing, the article proposes several efficient optimization methods including vertical-first cost aggregation, hybrid parallel processing, and hardware-friendly integral image. The article also presents a customizable system which provides both accurate and high-speed stereo matching for high-resolution images. The benefits of applying the optimization methods to the system are highlighted. With the aforesaid optimization and specific customization implemented on FPGA, the demonstrated system can process 47.6 fps (frames per second) and 129 fps for video size of 1920 × 1080 with a large disparity range of 256 and 1024 × 768 with a disparity range of 128, respectively. Our results are up to 1.64 times better than previous work in terms of Million Disparity Estimation per second (MDE/s). For accuracy, the 7.65&percnt; overall average error rate outperforms current work which can provide real-time processing with this high-resolution and large disparity range.

TinyStereo: A Tiny Coarse-to-Fine Framework for Vision-Based Depth Estimation on Embedded GPUs

Real-time Stereo Vision System Using Adaptive Weight Cost Aggregation Approach

A Depth Estimation Framework Based on Unsupervised Learning and Cross-Modal Translation

A Robust Monocular Depth Estimation Framework Based on Light-Weight ERF-Pspnet for Day-Night Driving Scenes

StereoVAE: A Lightweight Stereo-Matching System Using Embedded GPUs.

Stereo Matching Accelerator With Re-Computation Scheme and Data-Reused Pipeline for Autonomous Vehicles

On the Importance of Stereo for Accurate Depth Estimation: An Efficient Semi-Supervised Deep Neural Network Approach

Real-time Monocular Depth Estimation on Embedded Systems

Hardware Solution of Real-Time Depth Estimation Based on Stereo Vision

Accurate Real-Time Stereo Correspondence Using Intra- and Inter-Scanline Optimization

Real-Time Stereo Image Depth Estimation Network with Group-Wise L1 Distance for Edge Devices Towards Autonomous Driving

Efficient stereo matching on embedded GPUs with zero-means cross correlation

Anytime Stereo Image Depth Estimation on Mobile Devices

Hardware Acceleration for an Accurate Stereo Vision System Using Mini-Census Adaptive Support Region

Lightweight multi-scale convolutional neural network for real time stereo matching

FP-Stereo: Hardware-Efficient Stereo Vision for Embedded Applications

Fast Deep Stereo with 2D Convolutional Processing of Cost Signatures

FastDepth: Fast Monocular Depth Estimation on Embedded Systems

Real-Time Monocular Human Depth Estimation and Segmentation on Embedded Systems

Re-Parameterized Real-Time Stereo Matching Network Based on Mixed Cost Volumes Toward Autonomous Driving

An end-to-end stereo matching algorithm based on improved convolutional neural network