Abstract:Deep learning approaches have significantly contributed to recent progress in stereo matching. These deep stereo matching methods are usually based on supervised training, which requires a large amount of high-quality ground-truth depth map annotations that are expensive to collect. Furthermore, only a limited quantity of stereo vision training data are currently available, obtained either by active sensors (Lidar and ToF cameras) or through computer graphics simulations and not meeting requirements for deep supervised training. Here, we propose a novel deep stereo approach called the "self-supervised multiscale adversarial regression network (SMAR-Net)," which relaxes the need for ground-truth depth maps for training. Specifically, we design a two-stage network. The first stage is a disparity regressor, in which a regression network estimates disparity values from stacked stereo image pairs. Stereo image stacking method is a novel contribution as it not only contains the spatial appearances of stereo images but also implies matching correspondences with different disparity values. In the second stage, a synthetic left image is generated based on the left–right consistency assumption. Our network is trained by minimizing a hybrid loss function composed of a content loss and an adversarial loss. The content loss minimizes the average warping error between the synthetic images and the real ones. In contrast to the generative adversarial loss, our proposed adversarial loss penalizes mismatches using multiscale features. This constrains the synthetic image and real image as being pixelwise identical instead of just belonging to the same distribution. Furthermore, the combined utilization of multiscale feature extraction in both the content loss and adversarial loss further improves the adaptability of SMAR-Net in ill-posed regions. Experiments on multiple benchmark datasets show that SMAR-Net outperforms the curr-nt state-of-the-art self-supervised methods and achieves comparable outcomes to supervised methods. The source code can be accessed at: https://github.com/Dawnstar8411/SMAR-Net.

Unsupervised Stereo Matching With Occlusion-Aware Loss

Unsupervised Occlusion-Aware Stereo Matching with Directed Disparity Smoothing

Geometry-based Occlusion-Aware Unsupervised Stereo Matching for Autonomous Driving

Occlusion Aware Stereo Matching Via Cooperative Unsupervised Learning

Stereo Matching by Self-supervision of Multiscopic Vision.

Robust stereo matching with surface normal prediction.

Self-Supervised Learning for Stereo Matching with Self-Improving Ability

Unsupervised Stereo Depth Estimation Refined by Perceptual Loss.

Adaptive Cost Volume Representation for Unsupervised High-resolution Stereo Matching

Improving Stereo Matching by Incorporating Geometry Prior into Convnet

A Semi-Supervised Monocular Stereo Matching Method

A Novel Cell Structure‐based Disparity Estimation for Unsupervised Stereo Matching

Adaptive Multi-Modal Cross-Entropy Loss for Stereo Matching

Sparse LiDAR Assisted Self-supervised Stereo Disparity Estimation

Stereo matching method based on high-precision occlusion-recovering and discontinuity-preserving

Pseudo-Stereo Inputs: A Solution to the Occlusion Challenge in Self-Supervised Stereo Matching

A unified and efficient semi-supervised learning framework for stereo matching

Multi-directional Broad Learning System for the Unsupervised Stereo Matching Method.

Self-Supervised Multiscale Adversarial Regression Network for Stereo Disparity Estimation

EHSS: an Efficient Hybrid-supervised Symmetric Stereo Matching Network

An end-to-end stereo matching algorithm based on improved convolutional neural network