Abstract:Deep learning approaches have significantly contributed to recent progress in stereo matching. These deep stereo matching methods are usually based on supervised training, which requires a large amount of high-quality ground-truth depth map annotations that are expensive to collect. Furthermore, only a limited quantity of stereo vision training data are currently available, obtained either by active sensors (Lidar and ToF cameras) or through computer graphics simulations and not meeting requirements for deep supervised training. Here, we propose a novel deep stereo approach called the "self-supervised multiscale adversarial regression network (SMAR-Net)," which relaxes the need for ground-truth depth maps for training. Specifically, we design a two-stage network. The first stage is a disparity regressor, in which a regression network estimates disparity values from stacked stereo image pairs. Stereo image stacking method is a novel contribution as it not only contains the spatial appearances of stereo images but also implies matching correspondences with different disparity values. In the second stage, a synthetic left image is generated based on the left–right consistency assumption. Our network is trained by minimizing a hybrid loss function composed of a content loss and an adversarial loss. The content loss minimizes the average warping error between the synthetic images and the real ones. In contrast to the generative adversarial loss, our proposed adversarial loss penalizes mismatches using multiscale features. This constrains the synthetic image and real image as being pixelwise identical instead of just belonging to the same distribution. Furthermore, the combined utilization of multiscale feature extraction in both the content loss and adversarial loss further improves the adaptability of SMAR-Net in ill-posed regions. Experiments on multiple benchmark datasets show that SMAR-Net outperforms the curr-nt state-of-the-art self-supervised methods and achieves comparable outcomes to supervised methods. The source code can be accessed at: https://github.com/Dawnstar8411/SMAR-Net.

Stereo Risk: A Continuous Modeling Approach to Stereo Matching

Segment-based Continuous Stereo Matching

Patchmatch Stereo++: Patchmatch Binocular Stereo with Continuous Disparity Optimization

EBStereo: Edge-Based Loss Function for Real-Time Stereo Matching

Robust stereo matching with surface normal prediction.

Deep Stereo Matching With Hysteresis Attention and Supervised Cost Volume Construction

Modeling Stereo-Confidence Out of the End-to-End Stereo-Matching Network via Disparity Plane Sweep

On the confidence of stereo matching in a deep-learning era: a quantitative evaluation

Unsupervised Stereo Matching With Occlusion-Aware Loss

Adaptive Cost Volume Representation for Unsupervised High-resolution Stereo Matching

Stereo Matching Using Multi-Level Cost Volume and Multi-Scale Feature Constancy

Continuous Symmetric Stereo with Adaptive Outlier Handling

Stereo Anything: Unifying Stereo Matching with Large-Scale Mixed Data

Practical Stereo Matching via Cascaded Recurrent Network with Adaptive Correlation

Exploiting Semantic and Boundary Information for Stereo Matching

Adaptive Multi-Modal Cross-Entropy Loss for Stereo Matching

Adaptive Unimodal Cost Volume Filtering for Deep Stereo Matching.

Self-Supervised Multiscale Adversarial Regression Network for Stereo Disparity Estimation

UGNet: Uncertainty aware geometry enhanced networks for stereo matching

Neural Markov Random Field for Stereo Matching

Uncertainty estimation for stereo matching based on evidential deep learning