Abstract:Self-supervised stereo matching holds great promise for application and research due to its independence from expensive labeled data. However, direct self-supervised stereo matching paradigms based on photometric loss functions have consistently struggled with performance issues due to the occlusion challenge. The crux of the occlusion challenge lies in the fact that the positions of occluded pixels consistently align with the epipolar search direction defined by the input stereo images, leading to persistent information loss and erroneous feedback at fixed locations during self-supervised training. In this work, we propose a simple yet highly effective pseudo-stereo inputs strategy to address the core occlusion challenge. This strategy decouples the input and feedback images, compelling the network to probabilistically sample information from both sides of the occluding objects. As a result, the persistent lack of information in the aforementioned fixed occlusion areas is mitigated. Building upon this, we further address feedback conflicts and overfitting issues arising from the strategy. By integrating these components, our method achieves stable and significant performance improvements compared to existing methods. Quantitative experiments are conducted to evaluate the performance. Qualitative experiments further demonstrate accurate disparity inference even at occluded regions. These results demonstrate a significant advancement over previous methods in the field of direct self-supervised stereo matching based on photometric loss. The proposed pseudo-stereo inputs strategy, due to its simplicity and effectiveness, has the potential to serve as a new paradigm for direct self-supervised stereo matching. Code is available at <a class="link-external link-https" href="https://github.com/qrzyang/Pseudo-Stereo" rel="external noopener nofollow">this https URL</a>.

An Algorithm for Single View Occlusion Area Detection in Binocular Stereo Matching.

Efficient Binocular Stereo Matching Based on Sad and Improved Census Transformation

Leveraging Front and Side Cues for Occlusion Handling in Monocular 3D Object Detection

Local Spatial Continuity Steered Sparse Representation for Occluded Face Recognition

Performance evaluation of feature detection and matching in stereo visual odometry.

Accurate Real-Time Stereo Correspondence Using Intra- and Inter-Scanline Optimization

Stereo matching method based on high-precision occlusion-recovering and discontinuity-preserving

High-Quality Depth Recovery Via Interactive Multi-view Stereo

Pseudo-Stereo Inputs: A Solution to the Occlusion Challenge in Self-Supervised Stereo Matching

Mixed Reality Depth Contour Occlusion Using Binocular Similarity Matching and Three-dimensional Contour Optimisation

SCALABLE AND OCCLUSION-AWARE MULTI-CUES CORRELATION FILTER FOR ROBUST STEREO VISUAL TRACKING

Real-Time Dense Reconstruction with Binocular Endoscopy Based on StereoNet and ORB-SLAM

A Point-Cloud-Based Multiview Stereo Algorithm for Free-Viewpoint Video

OMNET: Real-Time Stereo Matching with Unsupervised Occlusion Mask.

Parallax attention stereo matching network based on the improved group-wise correlation stereo network

Salient Object Detection and Classification for Stereoscopic Images

Segment-Based Disparity Refinement With Occlusion Handling for Stereo Matching

Robust Cost Volume Generation Method for Dense Stereo Matching in Endoscopic Scenarios

A Stereo Algorithm Using Edge-Based Orthogonal Dynamic Programming

Symmetric Pixel-Group Based Stereo Matching for Occlusion Handling

SVO-PL: Stereo Visual Odometry with Fusion of Points and Line Segments