Abstract:Domain of stereo vision is highly important in the fields of autonomous cars, video tolling, robotics, and aerial surveys. The specific feature of this domain is that we should handle not only the pixel-by-pixel 2D processing in one image but also the 3D processing for depth estimation by comparing information about a scene from several images with different perspectives. This feature brings challenges to memory resource utilization, because an extra dimension of data has to be buffered. Due to the memory limitation, few of previous stereo vision implementations provide both accurate and high-speed processing for high-resolution images at the same time. To achieve domain-specific acceleration for stereo vision, the memory limitation has to be addressed. This article uses a Mini-Census ADaptive Support Region (MCADSR) stereo matching algorithm as a case study due to its high accuracy and representative operations in this domain. To relieve the memory limitation and achieve high-speed processing, the article proposes several efficient optimization methods including vertical-first cost aggregation, hybrid parallel processing, and hardware-friendly integral image. The article also presents a customizable system which provides both accurate and high-speed stereo matching for high-resolution images. The benefits of applying the optimization methods to the system are highlighted. With the aforesaid optimization and specific customization implemented on FPGA, the demonstrated system can process 47.6 fps (frames per second) and 129 fps for video size of 1920 × 1080 with a large disparity range of 256 and 1024 × 768 with a disparity range of 128, respectively. Our results are up to 1.64 times better than previous work in terms of Million Disparity Estimation per second (MDE/s). For accuracy, the 7.65&percnt; overall average error rate outperforms current work which can provide real-time processing with this high-resolution and large disparity range.

Image Processing Memory Optimization for Multi-camera Based Advanced Driver Assistance Systems

Towards Memory-Efficient Inference in Edge Video Analytics

Hardware Acceleration for an Accurate Stereo Vision System Using Mini-Census Adaptive Support Region

A Many-Ported and Shared Memory Architecture for High-Performance ADAS SoCs

Optimizing Monocular Driving Assistance for Real-Time Processing on Jetson AGX Xavier

Multi-task ADAS system on FPGA

IM3A: Boosting Deep Neural Network Efficiency via In-Memory Addressing-Assisted Acceleration

AiDAC: A Low-Cost In-Memory Computing Architecture with All-Analog Multi-Bit Compute and Interconnect

Accelerating GPU Message Communication for Autonomous Navigation Systems.

Large-Scale Bandwidth and Power Optimization for Multi-Modal Edge Intelligence Autonomous Driving

High Quality Image Processing System for ADAS

HARP: Hierarchical Attention Oriented Region-Based Processing for High-Performance Computation in Vision Sensor

A Power Efficient Architecture with Optimized Parallel Memory Accessing for Feature Generation

Low-Latency Edge Video Analytics for On-Road Perception of Autonomous Ground Vehicles

P2M-DeTrack: Processing-in-Pixel-in-Memory for Energy-efficient and Real-Time Multi-Object Detection and Tracking

Memory Optimization for Embedded Systems Running H.264/AVC Video Encoder

Memory Access Optimization of a Neural Network Accelerator Based on Memory Controller

Optimized Mapping Of Pixels Into Memory For H.264/Avc Decoding

Efficient Hardware Optimization Strategies For Deep Neural Networks Acceleration Chip

An Energy-Efficient Quantized and Regularized Training Framework for Processing-In-Memory Accelerators

Collaborative processing and data optimization of environmental perception technologies for autonomous vehicles