Abstract:Deep learning-based stereo matching methods have made remarkable progress in recent years. However, it is still a challenging task to achieve high accuracy in real time. In response to this challenge, we propose a Spatial Attention-Guided Upsampling network (SAGU-Net) for accurate and real-time stereo matching. First, a Spatial Attention-Guided Cost Volume Upsampling (SAG-CVU) module is proposed for upsampling the low-resolution cost volume, which calculates each upsampled matching cost as the sum of neighboring coarse costs under the guidance of spatial attention. Different from the recently popular coarse-to-fine (CTF) strategy that prefers upsampling the coarse disparity map, the low-resolution cost volume is upsampled by the SAG-CVU module which allows more raw information to propagate to subsequent procedures and can alleviate the problem of losing high-frequency information. To ensure fast running speed, a medium-resolution disparity map is directly regressed from the upsampled cost volume and then upsampled to full resolution with a Spatial Attention-Guided Disparity Map Upsampling (SAG-DMU) module. Unlike most CTF-based methods which usually build and aggregate narrow cost volumes iteratively until a full-resolution disparity map is obtained, the SAG-DMU module helps the proposed network avoid the iterative procedure to ensure fast running speed. In addition, we propose a simple yet effective gradient loss function that plays the role of a discontinuity-preserving regularizer, which further improves the overall accuracy, especially at depth discontinuities. These design choices lead to the proposed SAGU-Net which can obtain accurate results in real time. Extensive experimental results demonstrate that SAGU-Net and its variants outperform not only state-of-the-art real-time networks but also many accuracy-oriented models on multiple datasets.

Guided aggregation and disparity refinement for real-time stereo matching

Improved real-time three-dimensional stereo matching with local consistency

Stereo Matching Method with Integrated Geometric Encoding for Disparity Refinement

Accurate Real-Time Stereo Correspondence Using Intra- and Inter-Scanline Optimization

Efficient and Accurate Stereo Matching Via Guided Deformable Aggregation

MA-Stereo: Real-Time Stereo Matching Via Multi-Scale Attention Fusion and Spatial Error-Aware Refinement

Geometry and Context Guided Refinement for Stereo Matching

Stereo matching algorithm using improved guided filtering

Disparity Estimation Disparity Refinement Disparity Convolution Correlation Initial Disparity Residual Deconvolution Refined Disparity Reconstruction Error Shared Weights Skip Connection

Superpixel Guided Network for Three-Dimensional Stereo Matching

Real-time stereo matching with high accuracy via Spatial Attention-Guided Upsampling

UGNet: Uncertainty aware geometry enhanced networks for stereo matching

Ghrnet: Guided Hierarchical Refinement Network For Stereo Matching

GPDF-Net: geometric prior-guided stereo matching with disparity fusion refinement

Efficient Stereo Matching with Adaptive Disparity Range Scaling and Feature Calibration

Segment-Based Disparity Refinement With Occlusion Handling for Stereo Matching

CVCNet: Learning Cost Volume Compression for Efficient Stereo Matching

Cost Volume Aggregation in Stereo Matching Revisited: A Disparity Classification Perspective

Inter-Scale Similarity Guided Cost Aggregation for Stereo Matching

GA-Stereo: A Real-Time Stereo Network Based on the Gradient Flow Shunting Strategy and the Atrous Pyramid Network

Real-Time Accurate Stereo Matching Using Modified Two-Pass Aggregation and Winner-Take-All Guided Dynamic Programming