Abstract:Deep learning-based stereo matching methods have made remarkable progress in recent years. However, it is still a challenging task to achieve high accuracy in real time. In response to this challenge, we propose a Spatial Attention-Guided Upsampling network (SAGU-Net) for accurate and real-time stereo matching. First, a Spatial Attention-Guided Cost Volume Upsampling (SAG-CVU) module is proposed for upsampling the low-resolution cost volume, which calculates each upsampled matching cost as the sum of neighboring coarse costs under the guidance of spatial attention. Different from the recently popular coarse-to-fine (CTF) strategy that prefers upsampling the coarse disparity map, the low-resolution cost volume is upsampled by the SAG-CVU module which allows more raw information to propagate to subsequent procedures and can alleviate the problem of losing high-frequency information. To ensure fast running speed, a medium-resolution disparity map is directly regressed from the upsampled cost volume and then upsampled to full resolution with a Spatial Attention-Guided Disparity Map Upsampling (SAG-DMU) module. Unlike most CTF-based methods which usually build and aggregate narrow cost volumes iteratively until a full-resolution disparity map is obtained, the SAG-DMU module helps the proposed network avoid the iterative procedure to ensure fast running speed. In addition, we propose a simple yet effective gradient loss function that plays the role of a discontinuity-preserving regularizer, which further improves the overall accuracy, especially at depth discontinuities. These design choices lead to the proposed SAGU-Net which can obtain accurate results in real time. Extensive experimental results demonstrate that SAGU-Net and its variants outperform not only state-of-the-art real-time networks but also many accuracy-oriented models on multiple datasets.

SPNet: Learning Stereo Matching With Slanted Plane Aggregation

Asymmetric Cost Aggregation Network for Efficient Stereo Matching

Deep Stereo Matching with Explicit Cost Aggregation Sub-Architecture

HIERARCHICAL CONTEXT GUIDED AGGREGATION NETWORK FOR STEREO MATCHING

AANet: Adaptive Aggregation Network for Efficient Stereo Matching

Cost Volume Aggregation in Stereo Matching Revisited: A Disparity Classification Perspective

Initial Left Image R ight Image Hand-Designed Cost Aggregation Disparity Map Disparity Computation A ggregated C ost Volume Extracting Guidance Aggregation Guidance Generaing Aggregation Proposals Aggregation Proposals Fusion Aggregated Cost Volume Disparity Map Disparity Computation A : The Typical

Self-adaptive Multi-scale Aggregation Network for Stereo Matching.

Attention-guided aggregation stereo matching network

Inter-Scale Similarity Guided Cost Aggregation for Stereo Matching

Ghost-Stereo: GhostNet-based Cost Volume Enhancement and Aggregation for Stereo Matching Networks

Hierarchical and Multi-Level Cost Aggregation for Stereo Matching.

Multi-Level Context Ultra-Aggregation for Stereo Matching

Efficient and Accurate Stereo Matching Via Guided Deformable Aggregation

Region Separable Stereo Matching

HPA-Net: Hierarchical and Parallel Aggregation Network for Context Learning in Stereo Matching

Local Similarity Pattern and Cost Self-Reassembling for Deep Stereo Matching Networks

Edge-preserving Guided Filtering Based Cost Aggregation for Stereo Matching.

Multi-Scale Cost Volumes Cascade Network for Stereo Matching

SCV-Stereo: Learning Stereo Matching from a Sparse Cost Volume

Real-time stereo matching with high accuracy via Spatial Attention-Guided Upsampling