Abstract:Deep learning approaches have significantly contributed to recent progress in stereo matching. These deep stereo matching methods are usually based on supervised training, which requires a large amount of high-quality ground-truth depth map annotations that are expensive to collect. Furthermore, only a limited quantity of stereo vision training data are currently available, obtained either by active sensors (Lidar and ToF cameras) or through computer graphics simulations and not meeting requirements for deep supervised training. Here, we propose a novel deep stereo approach called the "self-supervised multiscale adversarial regression network (SMAR-Net)," which relaxes the need for ground-truth depth maps for training. Specifically, we design a two-stage network. The first stage is a disparity regressor, in which a regression network estimates disparity values from stacked stereo image pairs. Stereo image stacking method is a novel contribution as it not only contains the spatial appearances of stereo images but also implies matching correspondences with different disparity values. In the second stage, a synthetic left image is generated based on the left–right consistency assumption. Our network is trained by minimizing a hybrid loss function composed of a content loss and an adversarial loss. The content loss minimizes the average warping error between the synthetic images and the real ones. In contrast to the generative adversarial loss, our proposed adversarial loss penalizes mismatches using multiscale features. This constrains the synthetic image and real image as being pixelwise identical instead of just belonging to the same distribution. Furthermore, the combined utilization of multiscale feature extraction in both the content loss and adversarial loss further improves the adaptability of SMAR-Net in ill-posed regions. Experiments on multiple benchmark datasets show that SMAR-Net outperforms the curr-nt state-of-the-art self-supervised methods and achieves comparable outcomes to supervised methods. The source code can be accessed at: https://github.com/Dawnstar8411/SMAR-Net.

Image restoration and fault tolerance of stereo SLAM based on generative adversarial net

A robust stereo feature-aided semi-direct SLAM system

Structure Aggregation for Cross-Spectral Stereo Image Guided Denoising

A Robust Stereo Semi-direct SLAM System Based on Hybrid Pyramid

Persistent Stereo Visual Localization on Cross-Modal Invariant Map

Stereo SLAM in Dynamic Environments Using Semantic Segmentation

From Satellite to Ground: Satellite Assisted Visual Localization with Cross-view Semantic Matching

Robust Stereo Visual SLAM for Dynamic Environments With Moving Object

From Augmentation to Inpainting: Improving Visual SLAM With Signal Enhancement Techniques and GAN-Based Image Inpainting

MN-SLAM: Multi-networks Visual SLAM for Dynamic and Complicated Environments

Stereo and LiDAR Loosely Coupled SLAM Constrained Ground Detection

UGNet: Uncertainty aware geometry enhanced networks for stereo matching

GA-Stereo: A Real-Time Stereo Network Based on the Gradient Flow Shunting Strategy and the Atrous Pyramid Network

StereoNeuroBayesSLAM: A Neurobiologically Inspired Stereo Visual SLAM System Based on Direct Sparse Method

Self-Supervised Multiscale Adversarial Regression Network for Stereo Disparity Estimation

An Improved Visual SLAM Algorithm Based on Graph Neural Network

Superpixel Guided Network for Three-Dimensional Stereo Matching

3D LiDAR and Stereo Fusion using Stereo Matching Network with Conditional Cost Volume Normalization

EAI-Stereo: Error Aware Iterative Network for Stereo Matching

Toward Real World Stereo Image Super-Resolution via Hybrid Degradation Model and Discriminator for Implied Stereo Image Information

DOT-SLAM: A Stereo Visual Simultaneous Localization and Mapping (SLAM) System with Dynamic Object Tracking Based on Graph Optimization