Abstract:Self-supervised stereo matching holds great promise for application and research due to its independence from expensive labeled data. However, direct self-supervised stereo matching paradigms based on photometric loss functions have consistently struggled with performance issues due to the occlusion challenge. The crux of the occlusion challenge lies in the fact that the positions of occluded pixels consistently align with the epipolar search direction defined by the input stereo images, leading to persistent information loss and erroneous feedback at fixed locations during self-supervised training. In this work, we propose a simple yet highly effective pseudo-stereo inputs strategy to address the core occlusion challenge. This strategy decouples the input and feedback images, compelling the network to probabilistically sample information from both sides of the occluding objects. As a result, the persistent lack of information in the aforementioned fixed occlusion areas is mitigated. Building upon this, we further address feedback conflicts and overfitting issues arising from the strategy. By integrating these components, our method achieves stable and significant performance improvements compared to existing methods. Quantitative experiments are conducted to evaluate the performance. Qualitative experiments further demonstrate accurate disparity inference even at occluded regions. These results demonstrate a significant advancement over previous methods in the field of direct self-supervised stereo matching based on photometric loss. The proposed pseudo-stereo inputs strategy, due to its simplicity and effectiveness, has the potential to serve as a new paradigm for direct self-supervised stereo matching. Code is available at <a class="link-external link-https" href="https://github.com/qrzyang/Pseudo-Stereo" rel="external noopener nofollow">this https URL</a>.

Symmetric Pixel-Group Based Stereo Matching for Occlusion Handling

A Symmetric Patch-Based Correspondence Model for Occlusion Handling.

Segment-based Continuous Stereo Matching

Space-temporal stereo matching for dynamic scene

Efficient Binocular Stereo Matching Based on Sad and Improved Census Transformation

Object segmentation using stereo images

Segment-Based Disparity Refinement With Occlusion Handling for Stereo Matching

Stereo Correspondence with Occlusion Handling in a Symmetric Patch-Based Graph-Cuts Model.

Symmetric Segment-Based Stereo Matching Of Motion Blurred Images With Illumination Variations

Ambiguous Region Matching and Boundary Pixel Optimization for Stereo Computation

Stereo Matching with Space-Constrained Cost Aggregation and Segmentation-Based Disparity Refinement

Continuous Symmetric Stereo with Adaptive Outlier Handling

PM-PM: PatchMatch with Potts Model for Object Segmentation and Stereo Matching.

Pseudo-Stereo Inputs: A Solution to the Occlusion Challenge in Self-Supervised Stereo Matching

Simultaneous multi-body stereo and segmentation

A Fast Local Stereo Matching Insensitive to Illumination Conditions

Accurate Real-Time Stereo Correspondence Using Intra- and Inter-Scanline Optimization

Stereo superpixel: An iterative framework based on parallax consistency and collaborative optimization

Stereo matching method based on high-precision occlusion-recovering and discontinuity-preserving

Segmentation-based dense stereo matching algorithm for virtual view synthesis

Superpixel Alpha-Expansion and Normal Adjustment for Stereo Matching.