Abstract:Deep learning approaches have significantly contributed to recent progress in stereo matching. These deep stereo matching methods are usually based on supervised training, which requires a large amount of high-quality ground-truth depth map annotations that are expensive to collect. Furthermore, only a limited quantity of stereo vision training data are currently available, obtained either by active sensors (Lidar and ToF cameras) or through computer graphics simulations and not meeting requirements for deep supervised training. Here, we propose a novel deep stereo approach called the "self-supervised multiscale adversarial regression network (SMAR-Net)," which relaxes the need for ground-truth depth maps for training. Specifically, we design a two-stage network. The first stage is a disparity regressor, in which a regression network estimates disparity values from stacked stereo image pairs. Stereo image stacking method is a novel contribution as it not only contains the spatial appearances of stereo images but also implies matching correspondences with different disparity values. In the second stage, a synthetic left image is generated based on the left–right consistency assumption. Our network is trained by minimizing a hybrid loss function composed of a content loss and an adversarial loss. The content loss minimizes the average warping error between the synthetic images and the real ones. In contrast to the generative adversarial loss, our proposed adversarial loss penalizes mismatches using multiscale features. This constrains the synthetic image and real image as being pixelwise identical instead of just belonging to the same distribution. Furthermore, the combined utilization of multiscale feature extraction in both the content loss and adversarial loss further improves the adaptability of SMAR-Net in ill-posed regions. Experiments on multiple benchmark datasets show that SMAR-Net outperforms the curr-nt state-of-the-art self-supervised methods and achieves comparable outcomes to supervised methods. The source code can be accessed at: https://github.com/Dawnstar8411/SMAR-Net.

SRC-Disp: Synthetic-Realistic Collaborative Disparity Learning for Stereo Matching

Practical Stereo Matching via Cascaded Recurrent Network with Adaptive Correlation

Synthetic-to-Real Domain Adaptation Joint Spatial Feature Transform for Stereo Matching

Stacking Learning with Coalesced Cost Filtering for Accurate Stereo Matching

Adaptive Disparity Computation Using Local and Non-Local Cost Aggregations

Stereo matching from monocular images using feature consistency

Stereo Matching Method with Integrated Geometric Encoding for Disparity Refinement

Self-Supervised Multiscale Adversarial Regression Network for Stereo Disparity Estimation

AdaStereo: An Efficient Domain-Adaptive Stereo Matching Approach

A Disparity Refinement Framework for Learning-based Stereo Matching Methods in Cross-domain Setting for Laparoscopic Images

Local Similarity Pattern and Cost Self-Reassembling for Deep Stereo Matching Networks

Disparity Distribution Equalization: an Effective Data Enhancement for Stereo Matching

Few-Shot Stereo Matching with High Domain Adaptability Based on Adaptive Recursive Network

MSDC-Net: Multi-Scale Dense and Contextual Networks for Automated Disparity Map for Stereo Matching

End-to-End Stereo Matching Network with Local Adaptive Awareness

Guided aggregation and disparity refinement for real-time stereo matching

Improved real-time three-dimensional stereo matching with local consistency

End-to-End Learning of Multi-scale Convolutional Neural Network for Stereo Matching

SR-Stereo & DAPE: Stepwise Regression and Pre-trained Edges for Practical Stereo Matching

Stereo Matching Using Multi-Level Cost Volume and Multi-Scale Feature Constancy

Real-Time Stereo Matching Networks Based on Multi-Modal and Sparse-Dense Fusion