Abstract:Robot vision technology based on binocular vision holds tremendous potential for development in various fields, including 3D scene reconstruction, target detection, and autonomous driving. However, current binocular vision methods used in robotics engineering have limitations such as high costs, complex algorithms, and low reliability of the generated disparity map in different scenes. To overcome these challenges, a cross-domain stereo matching algorithm for binocular vision based on transfer learning was proposed in this paper, named Cross-Domain Adaptation and Transfer Learning Network (Ct-Net), which has shown valuable results in multiple robot scenes. First, this paper introduces a General Feature Extractor to extract rich general feature information for domain adaptive stereo matching tasks. Then, a feature adapter is used to adapt the general features to the stereo matching network. Furthermore, a Domain Adaptive Cost Optimization Module is designed to optimize the matching cost. A disparity score prediction module was also embedded to adaptively adjust the search range of disparity and optimize the cost distribution. The overall framework was trained using a phased strategy, and ablation experiments were conducted to verify the effectiveness of the training strategy. Compared with the prototype PSMNet, on KITTI 2015 benchmark, the 3PE-fg of Ct-Net in all regions and non-occluded regions decreased by 19.3 and 21.1% respectively, meanwhile, on the Middlebury dataset, the proposed algorithm improves the sample error rate at least 28.4%, which is the Staircase sample. The quantitative and qualitative results obtained from Middlebury, Apollo, and other datasets demonstrate that Ct-Net significantly improves the cross-domain performance of stereo matching. Stereo matching experiments in real-world scenes have shown that it can effectively address visual tasks in multiple scenes.

GraftNet: Towards Domain Generalized Stereo Matching with a Broad-Spectrum and Task-Oriented Feature

Matching-space Stereo Networks for Cross-domain Generalization

Local Similarity Pattern and Cost Self-Reassembling for Deep Stereo Matching Networks

Self-adaptive Multi-scale Aggregation Network for Stereo Matching.

Deep spatial and discriminative feature enhancement network for stereo matching

An application of stereo matching algorithm based on transfer learning on robots in multiple scenes

A Multimodal-Based Feature Generalization Model for Binocular Stereo Matching

CGI-Stereo: Accurate and Real-Time Stereo Matching via Context and Geometry Interaction

AANet: Adaptive Aggregation Network for Efficient Stereo Matching

Revisiting Domain Generalized Stereo Matching Networks from a Feature Consistency Perspective

SA-Net: Scene-Aware Network for Cross-domain Stereo Matching

GA-Stereo: A Real-Time Stereo Network Based on the Gradient Flow Shunting Strategy and the Atrous Pyramid Network

Stereo sample generation‐based domain generalization network for stereo matching

Multi-Scale Cost Volumes Cascade Network for Stereo Matching

Stereo Matching Using Multi-Level Cost Volume and Multi-Scale Feature Constancy

Practical Stereo Matching via Cascaded Recurrent Network with Adaptive Correlation

Rethinking the Key Factors for the Generalization of Remote Sensing Stereo Matching Networks

A Joint 2D-3D Complementary Network for Stereo Matching

AdaStereo: An Efficient Domain-Adaptive Stereo Matching Approach

Domain Generalized Stereo Matching Via Hierarchical Visual Transformation

Global Matching-Optimization Network for Stereo Depth Estimation