Abstract:Stereo matching has become a key technique for 3D environment perception in intelligent vehicles. For a considerable time, convolutional neural networks (CNNs) have remained the mainstream choice for feature extraction in this domain. Nonetheless, there is a growing consensus that the existing paradigm should evolve towards vision foundation models (VFM), particularly those developed based on vision Transformers (ViTs) and pre-trained through self-supervision on extensive, unlabeled datasets. While VFMs are adept at extracting informative, general-purpose visual features, specifically for dense prediction tasks, their performance often lacks in geometric vision tasks. This study serves as the first exploration of a viable approach for adapting VFMs to stereo matching. Our ViT adapter, referred to as ViTAS, is constructed upon three types of modules: spatial differentiation, patch attention fusion, and cross-attention. The first module initializes feature pyramids, while the latter two aggregate stereo and multi-scale contextual information into fine-grained features, respectively. ViTAStereo, which combines ViTAS with cost volume-based stereo matching back-end processes, achieves the top rank on the KITTI Stereo 2012 dataset and outperforms the second-best network StereoBase by approximately 7.9% in terms of the percentage of error pixels, with a tolerance of 3 pixels. Additional experiments across diverse scenarios further demonstrate its superior generalizability compared to all other state-of-the-art approaches. We believe this new paradigm will pave the way for the next generation of stereo matching networks.

Visually Imbalanced Stereo Matching

Human-Computer Interaction System Based on Binocular Stereo Vision

Efficient Binocular Stereo Matching Based on Sad and Improved Census Transformation

Unconstrained Self-Calibration of Stereo Camera on Visually Impaired Assistance Devices.

Stereo Matching Algorithm Based on Binocular Vision

Binocular Stereo Vision Technology Based on Human Visual Characteristics

Stereo matching from monocular images using feature consistency

Graph-based Stereo Matching by Incorporating Monocular Cues

Stereo Matching: Fundamentals, State-of-the-Art, and Existing Challenges

Stereo Matching Using Pixel Classification and Dual-Weighted Guided Filter

Stereo Matching by Self-supervision of Multiscopic Vision.

Stereo Anything: Unifying Stereo Matching with Large-Scale Mixed Data

An accurate stereo matching method based on color segments and edges

Temporally Consistent Stereo Matching

Stereo matching method based on high-precision occlusion-recovering and discontinuity-preserving

Single View Stereo Matching

Progress in Binocular Stereo Matching Algorithms

A New Principle toward Robust Matching in Human-like Stereovision

Binocular Stereo Matching Method Based On Parallax Waveform Analysis And Conjoint Measurement Of Saliency Map

On the Stereoscopic Composition of Wide Baseline Stereo Pairs

Playing to Vision Foundation Model's Strengths in Stereo Matching