Abstract:Self-supervised stereo matching holds great promise for application and research due to its independence from expensive labeled data. However, direct self-supervised stereo matching paradigms based on photometric loss functions have consistently struggled with performance issues due to the occlusion challenge. The crux of the occlusion challenge lies in the fact that the positions of occluded pixels consistently align with the epipolar search direction defined by the input stereo images, leading to persistent information loss and erroneous feedback at fixed locations during self-supervised training. In this work, we propose a simple yet highly effective pseudo-stereo inputs strategy to address the core occlusion challenge. This strategy decouples the input and feedback images, compelling the network to probabilistically sample information from both sides of the occluding objects. As a result, the persistent lack of information in the aforementioned fixed occlusion areas is mitigated. Building upon this, we further address feedback conflicts and overfitting issues arising from the strategy. By integrating these components, our method achieves stable and significant performance improvements compared to existing methods. Quantitative experiments are conducted to evaluate the performance. Qualitative experiments further demonstrate accurate disparity inference even at occluded regions. These results demonstrate a significant advancement over previous methods in the field of direct self-supervised stereo matching based on photometric loss. The proposed pseudo-stereo inputs strategy, due to its simplicity and effectiveness, has the potential to serve as a new paradigm for direct self-supervised stereo matching. Code is available at <a class="link-external link-https" href="https://github.com/qrzyang/Pseudo-Stereo" rel="external noopener nofollow">this https URL</a>.

Stereo Matching and Occlusion Detection with Integrity and Illusion Sensitivity

Space-temporal stereo matching for dynamic scene

Variational Stereo Matching with Left Right Consistency Constraint

Stereo matching method based on high-precision occlusion-recovering and discontinuity-preserving

Efficient Binocular Stereo Matching Based on Sad and Improved Census Transformation

Dynamic Limited-Search Matching Algorithm Based on Geometric Properties of Stereo Imaging

Stereo Matching Via Disparity Estimation And Surface Modeling

Outlier detection and disparity refinement in stereo matching

A Fast Local Stereo Matching Insensitive to Illumination Conditions

Symmetric Pixel-Group Based Stereo Matching for Occlusion Handling

Continuous Symmetric Stereo with Adaptive Outlier Handling

An Efficient Stereo Disparity Estimation Method Based on Region Information

Segment-Based Disparity Refinement With Occlusion Handling for Stereo Matching

Accurate Real-Time Stereo Correspondence Using Intra- and Inter-Scanline Optimization

An Interleaving Updating Framework of Disparity and Confidence Map for Stereo Matching

Temporally Consistent Stereo Matching

Pseudo-Stereo Inputs: A Solution to the Occlusion Challenge in Self-Supervised Stereo Matching

Stereo matching from monocular images using feature consistency

Ambiguous Region Matching and Boundary Pixel Optimization for Stereo Computation

Object segmentation using stereo images

Stereo Matching with Space-Constrained Cost Aggregation and Segmentation-Based Disparity Refinement