Abstract:Superpixel provides local pixel coherence and respects object boundary, which is beneficial to stereo matching. Recently, superpixel cues are introduced into deep stereo networks. These methods develop a superpixel-based sampling scheme to downsample input color images and upsample output disparity maps. However, in this way, the image details are inevitably lost in the downsampling and the upsampling process introduces errors in the final disparity as well. Besides, this mechanism further limits the possibility of utilizing larger and multi-scale superpixels, which are important to alleviate the matching ambiguity. To address these problems, a superpixel-guided stereo matching method (LSG-Stereo) is proposed, which explicitly exploits the feature and disparity consistency within multi-scale superpixels to improve disparity estimation. To effectively incorporate superpixel cues into a stereo matching network, two novel modules are designed, including Superpixel Attention Spatial Pyramid Pooling (SA-SPP) and Superpixel-Guided Refinement (SGR). The SA-SPP module takes advantage of the content-aware superpixel pooling to construct an adaptive spatial pooling pyramid for better feature extraction. The SGR module explicitly utilizes the disparity consistency over multi-scale superpixels to further refine the disparity estimation in details and ill-posed regions. The proposed method is evaluated on the Scene Flow dataset, KITTI 2012, and KITTI 2015 stereo benchmarks with comprehensive experiments. Experimental results demonstrate that our method can significantly improve the accuracy of stereo matching, especially in details, occlusions, and texture-less regions.

Semantic stereo: Integrating piecewise planar stereo with segmentation and classification

Segment-based Continuous Stereo Matching

Stixel Based Urban Scene Representation by Stereo Vision

SegStereo: Exploiting Semantic Information for Disparity Estimation

Object segmentation using stereo images

SemStereo: Semantic-Constrained Stereo Matching Network for Remote Sensing

Semantic Stereo for Incidental Satellite Images

PM-PM: PatchMatch with Potts Model for Object Segmentation and Stereo Matching.

Simultaneous multi-body stereo and segmentation

Stereo superpixel: An iterative framework based on parallax consistency and collaborative optimization

A stereo object segmentation algorithm based on disparity and temporal-spatial information

Ambiguous Region Matching and Boundary Pixel Optimization for Stereo Computation

Stereo Matching with Space-Constrained Cost Aggregation and Segmentation-Based Disparity Refinement

Geometry-Aware Instance Segmentation with Disparity Maps

Large-scale 3D Semantic Mapping Using Stereo Vision

Semantic Segmentation of Street Scenes Using Disparity Information

Effectively Modeling Piecewise Planar Urban Scenes Based on Structure Priors and CNN.

Superpixel Guided Network for Three-Dimensional Stereo Matching

Semantic Motion Segmentation for Urban Dynamic Scene Understanding.

Coordinating Multiple Disparity Proposals for Stereo Computation.

S3Net: Innovating Stereo Matching and Semantic Segmentation with a Single-Branch Semantic Stereo Network in Satellite Epipolar Imagery