A Greedy Approach for a Time-Dependent Scheduling Problem

Stanisław Gawiejnowicz,W. Kurc,Lidia Pankowska

DOI: https://doi.org/10.1007/3-540-48086-2_9

2001-09-09

Abstract:

What problem does this paper attempt to address?

Disparity Estimation Using Multilevel and Global Information

Yaru Zhang,Hongbin Lin,Chao Wu,Bin Liu

DOI: https://doi.org/10.1117/1.jei.28.2.023035

IF: 0.829

2019-01-01

Journal of Electronic Imaging

Abstract:Deep convolutional neural networks have shown prominent performance in stereo matching. However, current network architectures lack performance in exploiting context and global information to finding corresponding points in ill-posed regions. A stereo matching network without postprocessing is proposed to solve this problem. This network combines the improved multilevel feature pyramid pooling module with the light two-dimensional (2-D) convolution subnetwork to efficiently utilize multilevel information and global information. In the multilevel feature pyramid pooling module, the base image feature is extracted by cascading three small convolution filters. Features of a stereo image pair are calculated by hierarchically fusing and pooling features information of the same scale after using the residual network. Multilevel semantic information is fully utilized to improve the robustness of image feature representation in multilevel feature pyramid pooling module. In the light 2-D convolution subnetwork, low-level structural information is obtained from the target image by three concatenated convolution layers with small convolution filters. Low-level information is used to rectify matching cost with global view to improve matching accuracy. The experimental results on the Scene Flow dataset, the MPI Sintel dataset, and the Middlebury dataset show that the performance obtained by the proposed network can be improved in the ill-posed regions. Matching accuracy is competitive compared to other results obtained by end-to-end networks without postprocessing. (C) 2019 SPIE and IS&T
Efficient Binocular Stereo Matching Based on Sad and Improved Census Transformation

Yun Zhang,Wenxiang Chen,Han Liu,Jinhua Liu,Hui Du

DOI: https://doi.org/10.1109/icmlc48188.2019.8949324

2019-01-01

Abstract:Binocular stereo matching aims to obtain disparities from two very close views. Existing stereo matching methods may cause false matching when there are much image noise and disparity discontinuities. This paper proposes a novel binocular stereo matching algorithm based on SAD and improved Census transformation. We first perform improved Census transformation, and then we get the matching costs by combining SAD and improved Census transformation. Finally we cluster the matching costs and calculate the disparities. To generate better disparities, we further propose the improved bilateral and selective filters to enhance the accuracy of disparities. Experimental results show that our binocular stereo matching can produce more accurate and complete disparities, and it works well in complex scenes with irregular shapes and more objects, thus it has wide applications in stereoscopic image processing.
Modeling Stereo-Confidence Out of the End-to-End Stereo-Matching Network via Disparity Plane Sweep

Jae Young Lee,Woonghyun Ka,Jaehyun Choi,Junmo Kim

2024-01-23

Abstract:We propose a novel stereo-confidence that can be measured externally to various stereo-matching networks, offering an alternative input modality choice of the cost volume for learning-based approaches, especially in safety-critical systems. Grounded in the foundational concepts of disparity definition and the disparity plane sweep, the proposed stereo-confidence method is built upon the idea that any shift in a stereo-image pair should be updated in a corresponding amount shift in the disparity map. Based on this idea, the proposed stereo-confidence method can be summarized in three folds. 1) Using the disparity plane sweep, multiple disparity maps can be obtained and treated as a 3-D volume (predicted disparity volume), like the cost volume is constructed. 2) One of these disparity maps serves as an anchor, allowing us to define a desirable (or ideal) disparity profile at every spatial point. 3) By comparing the desirable and predicted disparity profiles, we can quantify the level of matching ambiguity between left and right images for confidence measurement. Extensive experimental results using various stereo-matching networks and datasets demonstrate that the proposed stereo-confidence method not only shows competitive performance on its own but also consistent performance improvements when it is used as an input modality for learning-based stereo-confidence methods.

Computer Vision and Pattern Recognition
Multi-Dimensional Cooperative Network for Stereo Matching

Wei Chen,Xiaogang Jia,Mingfei Wu,Zhengfa Liang

DOI: https://doi.org/10.1109/lra.2021.3130984

IF: 5.2

2022-01-01

IEEE Robotics and Automation Letters

Abstract:The dimensions of cost volumes and corresponding aggregation networks play a critical role in balancing the speed and accuracy for stereo matching. Current most 2D stereo networks based on 3D cost volumes are speedy but poor in precision, while 3D stereo networks with 4D volumes are slow but precise. To tackle this problem, we propose a novel stereo framework for combining 2D and 3D networks. The disparity range generated by the 2D network is used to guide the 3D network and reduce the computational complexity. Meanwhile, the 3D network can refine the rough disparity from coarse to fine. Through the cooperation of 2D and 3D networks, our method can achieve accurate stereo results at a fast speed. We evaluate our method on three popular public benchmark datasets. Experimental results from the KITTI official website show that, our method can achieve similar accuracy with other 3D stereo networks (PSMNet, GCNet, GwcNet, etc.) at a significantly faster speed. The ablation studies further demonstrate the facticity and reasonability of our propose method.

robotics
Stereo Matching Using Multi-Level Cost Volume and Multi-Scale Feature Constancy

Zhengfa Liang,Yulan Guo,Yiliu Feng,Wei Chen,Linbo Qiao,Li Zhou,Jianfeng Zhang,Hengzhu Liu

DOI: https://doi.org/10.1109/tpami.2019.2928550

IF: 23.6

2019-01-01

IEEE Transactions on Pattern Analysis and Machine Intelligence

Abstract:For CNNs based stereo matching methods, cost volumes play an important role in achieving good matching accuracy. In this paper, we present an end-to-end trainable convolution neural network to fully use cost volumes for stereo matching. Our network consists of three sub-modules, i.e., shared feature extraction, initial disparity estimation, and disparity refinement. Cost volumes are calculated at multiple levels using the shared features, and are used in both initial disparity estimation and disparity refinement sub-modules. To improve the efficiency of disparity refinement, multi-scale feature constancy is introduced to measure the correctness of the initial disparity in feature space. These sub-modules of our network are tightly-coupled, making it compact and easy to train. Moreover, we investigate the problem of developing a robust model to perform well across multiple datasets with different characteristics. We achieve this by introducing a two-stage finetuning scheme to gently transfer the model to target datasets. Specifically, in the first stage, the model is finetuned using both a large synthetic dataset and the target datasets with a relatively large learning rate, while in the second stage the model is trained using only the target datasets with a small learning rate. The proposed method is tested on several benchmarks including the Middlebury 2014, KITTI 2015, ETH3D 2017, and SceneFlow datasets. Experimental results show that our method achieves the state-of-the-art performance on all the datasets. The proposed method also won the 1st prize on the Stereo task of Robust Vision Challenge 2018.
A Joint 2D-3D Complementary Network for Stereo Matching

Xiaogang Jia,Wei Chen,Zhengfa Liang,Xin Luo,Mingfei Wu,Chen Li,Yulin He,Yusong Tan,Libo Huang

DOI: https://doi.org/10.3390/s21041430

IF: 3.9

2021-02-18

Sensors

Abstract:Stereo matching is an important research field of computer vision. Due to the dimension of cost aggregation, current neural network-based stereo methods are difficult to trade-off speed and accuracy. To this end, we integrate fast 2D stereo methods with accurate 3D networks to improve performance and reduce running time. We leverage a 2D encoder-decoder network to generate a rough disparity map and construct a disparity range to guide the 3D aggregation network, which can significantly improve the accuracy and reduce the computational cost. We use a stacked hourglass structure to refine the disparity from coarse to fine. We evaluated our method on three public datasets. According to the KITTI official website results, Our network can generate an accurate result in 80 ms on a modern GPU. Compared to other 2D stereo networks (AANet, DeepPruner, FADNet, etc.), our network has a big improvement in accuracy. Meanwhile, it is significantly faster than other 3D stereo networks (5× than PSMNet, 7.5× than CSN and 22.5× than GANet, etc.), demonstrating the effectiveness of our method.

engineering, electrical & electronic,chemistry, analytical,instruments & instrumentation
Deep Stereo Matching With Hysteresis Attention and Supervised Cost Volume Construction

Kai Zeng,Yaonan Wang,Jianxu Mao,Caiping Liu,Weixing Peng,Yin Yang

DOI: https://doi.org/10.1109/TIP.2021.3135485

Abstract:Stereo matching disparity prediction for rectified image pairs is of great importance to many vision tasks such as depth sensing and autonomous driving. Previous work on the end-to-end unary trained networks follows the pipeline of feature extraction, cost volume construction, matching cost aggregation, and disparity regression. In this paper, we propose a deep neural network architecture for stereo matching aiming at improving the first and second stages of the matching pipeline. Specifically, we show a network design inspired by hysteresis comparator in the circuit as our attention mechanism. Our attention module is multiple-block and generates an attentive feature directly from the input. The cost volume is constructed in a supervised way. We try to use data-driven to find a good balance between informativeness and compactness of extracted feature maps. The proposed approach is evaluated on several benchmark datasets. Experimental results demonstrate that our method outperforms previous methods on SceneFlow, KITTI 2012, and KITTI 2015 datasets.
An efficient and accurate multi-level cascaded recurrent network for stereo matching

Ziyu Zhong,Xiuze Yang,Xiubian Pan,Wei Guan,Ke Liang,Jing Li,Xiaolan Liao,Shuo Wang

DOI: https://doi.org/10.1038/s41598-024-57321-6

IF: 4.6

2024-04-10

Scientific Reports

Abstract:With the advent of Transformer-based convolutional neural networks, stereo matching algorithms have achieved state-of-the-art accuracy in disparity estimation. Nevertheless, this method requires much model inference time, which is the main reason limiting its application in many vision tasks and robots. Facing the trade-off problem between accuracy and efficiency, this paper proposes an efficient and accurate multi-level cascaded recurrent network, LMCR-Stereo. To recover the detailed information of stereo images more accurately, we first design a multi-level network to update the difference values in a coarse-to-fine recurrent iterative manner. Then, we propose a new pair of slow-fast multi-stage superposition inference structures to accommodate the differences between different scene data. Besides, to ensure better disparity estimation accuracy with faster model inference speed, we introduce a pair of adaptive and lightweight group correlation layers to reduce the impact of erroneous rectification and significantly improve model inference speed. The experimental results show that the proposed approach achieves a competitive disparity estimation accuracy with a faster model inference speed than the current state-of-the-art methods. Notably, the model inference speed of the proposed approach is improved by 46.0% and 50.4% in the SceneFlow test set and Middlebury benchmark, respectively.

multidisciplinary sciences
A Fast Stereo Matching Network with Multi-Cross Attention

Ming Wei,Ming Zhu,Yi Wu,Jiaqi Sun,Jiarong Wang,Changji Liu

DOI: https://doi.org/10.3390/s21186016

IF: 3.9

2021-09-08

Sensors

Abstract:Stereo matching networks based on deep learning are widely developed and can obtain excellent disparity estimation. We present a new end-to-end fast deep learning stereo matching network in this work that aims to determine the corresponding disparity from two stereo image pairs. We extract the characteristics of the low-resolution feature images using the stacked hourglass structure feature extractor and build a multi-level detailed cost volume. We also use the edge of the left image to guide disparity optimization and sub-sample with the low-resolution data, ensuring excellent accuracy and speed at the same time. Furthermore, we design a multi-cross attention model for binocular stereo matching to improve the matching accuracy and achieve end-to-end disparity regression effectively. We evaluate our network on Scene Flow, KITTI2012, and KITTI2015 datasets, and the experimental results show that the speed and accuracy of our method are excellent.

engineering, electrical & electronic,chemistry, analytical,instruments & instrumentation
HLocalExp-CM: confidence map by hierarchical local expansion moves for accurate stereo matching

Xianjing Cheng,Yong Zhao,Weiping Zhu,Zhijun Hu,Xiaomin Yu,Wenbang Yang,Ren Qian

DOI: https://doi.org/10.1117/1.JEI.31.1.013003

IF: 0.829

2022-01-01

Journal of Electronic Imaging

Abstract:We present a stereo matching approach referred to as HLocalExp-CM by exploiting the hierarchical local contextual information and a confidence map based on a new grid structure. The proposed approach preserves fine depth edges and extracts accurate disparities in weak texture, textureless, and repeated texture regions. The proposed approach adopts a two-stage optimization strategy. In the framework of first stage, a multiresolution cost aggregation is minimized to reduce the search space of the disparity plane of each pixel. The second stage iteratively optimizes the confidence map and a global energy function to progressively improve the disparity accuracy for each pixel. The confidence map is estimated through classifying the pixels into distinctive and ambiguous ones by computing the decreasing rate of the multiresolution cost aggregation and then performs a spatial propagation and plane refinement for the update of the disparity of each pixel, thereby successfully eliminating the ambiguity of nondistinctive pixels. The global energy function based on a pairwise Markov random field uses cross-scale cost aggregation for taking advantage of context information of objects in different scenarios on local grid regions, which is different from the deep learning technique uses convolution layers extracting the context information. The proposed approach is evaluated on Middlebury benchmark V3, and is ranked first based on "bad 2.0 all metric," a widely used criterion for the evaluation of stereo images, while the eighth place on "bad 2.0 nonocc metric" (recorded on July 24, 2021). (C) 2022 SPIE and IS&T
Practical Stereo Matching via Cascaded Recurrent Network with Adaptive Correlation

Jiankun Li,Peisen Wang,Pengfei Xiong,Tao Cai,Ziwei Yan,Lei Yang,Jiangyu Liu,Haoqiang Fan,Shuaicheng Liu

DOI: https://doi.org/10.48550/arXiv.2203.11483

2022-03-22

Computer Vision and Pattern Recognition

Abstract:With the advent of convolutional neural networks, stereo matching algorithms have recently gained tremendous progress. However, it remains a great challenge to accurately extract disparities from real-world image pairs taken by consumer-level devices like smartphones, due to practical complicating factors such as thin structures, non-ideal rectification, camera module inconsistencies and various hard-case scenes. In this paper, we propose a set of innovative designs to tackle the problem of practical stereo matching: 1) to better recover fine depth details, we design a hierarchical network with recurrent refinement to update disparities in a coarse-to-fine manner, as well as a stacked cascaded architecture for inference; 2) we propose an adaptive group correlation layer to mitigate the impact of erroneous rectification; 3) we introduce a new synthetic dataset with special attention to difficult cases for better generalizing to real-world scenes. Our results not only rank 1st on both Middlebury and ETH3D benchmarks, outperforming existing state-of-the-art methods by a notable margin, but also exhibit high-quality details for real-life photos, which clearly demonstrates the efficacy of our contributions.
Temporally Consistent Stereo Matching

Jiaxi Zeng,Chengtang Yao,Yuwei Wu,Yunde Jia

2024-07-17

Abstract:Stereo matching provides depth estimation from binocular images for downstream applications. These applications mostly take video streams as input and require temporally consistent depth maps. However, existing methods mainly focus on the estimation at the single-frame level. This commonly leads to temporally inconsistent results, especially in ill-posed regions. In this paper, we aim to leverage temporal information to improve the temporal consistency, accuracy, and efficiency of stereo matching. To achieve this, we formulate video stereo matching as a process of temporal disparity completion followed by continuous iterative refinements. Specifically, we first project the disparity of the previous timestamp to the current viewpoint, obtaining a semi-dense disparity map. Then, we complete this map through a disparity completion module to obtain a well-initialized disparity map. The state features from the current completion module and from the past refinement are fused together, providing a temporally coherent state for subsequent refinement. Based on this coherent state, we introduce a dual-space refinement module to iteratively refine the initialized result in both disparity and disparity gradient spaces, improving estimations in ill-posed regions. Extensive experiments demonstrate that our method effectively alleviates temporal inconsistency while enhancing both accuracy and efficiency.

Computer Vision and Pattern Recognition
End-to-End Edge-Guided Multi-Scale Matching Network for Optical Satellite Stereo Image Pairs

Yixin Luo,Hao Wang,Xiaolei Lv

DOI: https://doi.org/10.3390/rs16050882

IF: 5

2024-03-03

Remote Sensing

Abstract:Acquiring disparity maps by dense stereo matching is one of the most important methods for producing digital surface models. However, the characteristics of optical satellite imagery, including significant occlusions and long baselines, increase the challenges of dense matching. In this study, we propose an end-to-end edge-guided multi-scale matching network (EGMS-Net) tailored for optical satellite stereo image pairs. Using small convolutional filters and residual blocks, the EGMS-Net captures rich high-frequency signals during the initial feature extraction phase. Subsequently, pyramid features are derived through efficient down-sampling and consolidated into cost volumes. To regularize these cost volumes, we design a top–down multi-scale fusion network that integrates an attention mechanism. Finally, we innovate the use of trainable guided filter layers in disparity refinement to improve edge detail recovery. The network is trained and evaluated using the Urban Semantic 3D and WHU-Stereo datasets, with subsequent analysis of the disparity maps. The results show that the EGMS-Net provides superior results, achieving endpoint errors of 1.515 and 2.459 pixels, respectively. In challenging scenarios, particularly in regions with textureless surfaces and dense buildings, our network consistently delivers satisfactory matching performance. In addition, EGMS-Net reduces training time and increases network efficiency, improving overall results.

environmental sciences,imaging science & photographic technology,remote sensing,geosciences, multidisciplinary
Stereo Matching by Self-supervision of Multiscopic Vision.

Weihao Yuan,Yazhan Zhang,Bingkun Wu,Siyu Zhu,Ping Tan,Michael Yu Wang,Qifeng Chen

DOI: https://doi.org/10.1109/iros51168.2021.9636616

2021-01-01

Abstract:Self-supervised learning for depth estimation possesses several advantages over supervised learning. The benefits of no need for ground-truth depth, online fine-tuning, and better generalization with unlimited data attract researchers to seek self-supervised solutions. In this work, we propose a new self-supervised framework for stereo matching utilizing multiple images captured at aligned camera positions. A cross photometric loss, an uncertainty-aware mutual-supervision loss, and a new smoothness loss are introduced to optimize the network in learning disparity maps end-to-end without ground-truth depth information. To train this framework, we build a new multiscopic dataset consisting of synthetic images rendered by 3D engines and real images captured by real cameras. After being trained with only the synthetic images, our network can perform well in unseen outdoor scenes. Our experiment shows that our model obtains better disparity maps than previous unsupervised methods on the KITTI dataset and is comparable to supervised methods when generalized to unseen data. Our source code and dataset are available at https://sites.google.com/view/multiscopic.
End-to-End Learning of Multi-scale Convolutional Neural Network for Stereo Matching

Li Zhang,Quanhong Wang,Haihua Lu,Yong Zhao

DOI: https://doi.org/10.48550/arxiv.1906.10399

2019-01-01

Abstract:Deep neural networks have shown excellent performance in stereo matching task. Recently CNN-based methods have shown that stereo matching can be formulated as a supervised learning task. However, less attention is paid on the fusion of contextual semantic information and details. To tackle this problem, we propose a network for disparity estimation based on abundant contextual details and semantic information, called Multi-scale Features Network (MSFNet). First, we design a new structure to encode rich semantic information and fine-grained details by fusing multi-scale features. And we combine the advantages of element-wise addition and concatenation, which is conducive to merge semantic information with details. Second, a guidance mechanism is introduced to guide the network to automatically focus more on the unreliable regions. Third, we formulate the consistency check as an error map, obtained by the low stage features with fine-grained details. Finally, we adopt the consistency checking between the left feature and the synthetic left feature to refine the initial disparity. Experiments on Scene Flow and KITTI 2015 benchmark demonstrated that the proposed method can achieve the state-of-the-art performance.
Parallax attention stereo matching network based on the improved group-wise correlation stereo network

Xuefei Yu,Jinan Gu,Zedong Huang,Zhijie Zhang

DOI: https://doi.org/10.1371/journal.pone.0263735

IF: 3.7

2022-02-09

PLoS ONE

Abstract:Recent stereo matching methods, especially end-to-end deep stereo matching networks, have achieved remarkable performance in the fields of autonomous driving and depth sensing. However, state-of-the-art stereo algorithms, even with the deep neural network framework, still have difficulties at finding correct correspondences in near-range regions and object edge cues. To reinforce the precision of disparity prediction, in the present study, we propose a parallax attention stereo matching algorithm based on the improved group-wise correlation stereo network to learn the disparity content from a stereo correspondence, and it supports end-to-end predictions of both disparity map and edge map. Particular, we advocate for a parallax attention module in three-dimensional (disparity, height and width) level, which structure ensures high-precision estimation by improving feature expression in near-range regions. This is critical for computer vision tasks and can be utilized in several existing models to enhance their performance. Moreover, in order to making full use of the edge information learned by two-dimensional feature extraction network, we propose a novel edge detection branch and multi-featured integration cost volume. It is demonstrated that based on our model, edge detection project is conducive to improve the accuracy of disparity estimation. Our method achieves better results than previous works on both Scene Flow and KITTI datasets.
EAI-Stereo: Error Aware Iterative Network for Stereo Matching

Haoliang Zhao,Huizhou Zhou,Yongjun Zhang,Yong Zhao,Yitong Yang,Ting Ouyang

DOI: https://doi.org/10.1007/978-3-031-26319-4_1

2023-01-01

Abstract:Current state-of-the-art stereo algorithms use a 2D CNN to extract features and then form a cost volume, which is fed into the following cost aggregation and regularization module composed of 2D or 3D CNNs. However, a large amount of high-frequency information like texture, color variation, sharp edge etc. is not well exploited during this process, which leads to relatively blurry and lacking detailed disparity maps. In this paper, we aim at making full use of the high-frequency information from the original image. Towards this end, we propose an error-aware refinement module that incorporates high-frequency information from the original left image and allows the network to learn error correction capabilities that can produce excellent subtle details and sharp edges. In order to improve the data transfer efficiency between our iterations, we propose the Iterative Multiscale Wide-LSTM Network which could carry more semantic information across iterations. We demonstrate the efficiency and effectiveness of our method on KITTI 2015, Middle-bury, and ETH3D. At the time of writing this paper, EAI-Stereo ranks 1st on the Middlebury leaderboard and 1st on the ETH3D Stereo benchmark for 50% quantile metric and second for 0.5px error rate among all published methods. Our model performs well in cross-domain scenarios and outperforms current methods specifically designed for generalization. Code is available at https://github.com/David- Zhao-1997/EAI-Stereo.
Multi-Scale Cost Volumes Cascade Network for Stereo Matching

Xiaogang Jia,Wei Chen,Chen Li Zhengfa Liang,Mingfei Wu,Yusong Tan,Libo Huang

DOI: https://doi.org/10.1109/icra48506.2021.9560864

2021-01-01

Abstract:Stereo matching is essential for robot navigation. However, the accuracy of current widely used traditional methods is low, while methods based on CNN need expensive computational cost and running time. This is because different cost volumes play a crucial role in balancing speed and accuracy. Thus we propose MSCVNet, which combines traditional methods and neural networks to improve the quality of cost volume. Concretely, our network first generates multiple 3D cost volumes with different resolutions and then uses 2D convolutions to construct a novel cascade hourglass network for cost aggregation. Meanwhile, we design an algorithm to distinguish and calculate the loss for discontinuous areas of the disparity result. According to the KITTI official website, our network is much faster than most top-performing methods (24than CSPN, 44than GANet, etc.). Meanwhile, compared to traditional methods (SPS-St, SGM) and other real-time stereo matching networks (Fast DS-CS, DispNetC, and RTSNet, etc.), our network achieves a big improvement in accuracy, demonstrating the feasibility and capability of the proposed method.
Multi-path Feature Mining Network for Stereo Matching

Quanhong Wang,Li Zhang,Hai Xu,Haiwei Sang,Zhao Yong

DOI: https://doi.org/10.1109/iciea.2019.8834088

2019-01-01

Abstract:Many existing networks for disparity estimation have attained satisfactory performance, but it still is a challenge to gain high-quality disparities for the inherently ill-posed regions and improve detail in disparity map. To tackle this problem, we propose a Multi-path Feature Mining Network(MFMNet), which contains two sub-networks: initial disparity estimation network and disparity refinement network. Specially, to generate high-quality disparity with rich detail, we specially design a initial disparity estimation network with multi-scale feature fusion and richer feature extraction module to capture context information and gain more reliable features. Furthermore, we propose a disparity refinement network with atrous spatial pyramid pooling to enhance disparity refinement performance. In this work, we adopt one-dimension single direction(1SD) correlation to calculate matching cost rather than the 1D correlation used widely. During training, we adopt a novel loss function strategy to perform online hard point mining, which enhance the overall network performance. Experimentation shows that our multi-path feature mining network provides state-of-the-art performance on FlyingThings3D and KITTI datasets.
Cascaded multi-scale and multi-dimension convolutional neural network for stereo matching.

Haihua Lu,Hai Xu,Li Zhang,Yong Zhao

DOI: https://doi.org/10.1109/VCIP.2018.8698637

2018-01-01

Abstract:Convolutional neural networks (CNN) have been shown to perform better than the conventional stereo algorithms for stereo estimation. Numerous CNN algorithms focus on the pixel-wise matching cost computation, which is the important building block for many state-of-the-art algorithms. However, these architectures are limited to small and single scale receptive fields and use traditional methods for cost aggregation or even ignore cost aggregation. In this paper, we propose a novel architecture called cascaded multi-scale and multi-dimension network (MSMD) to take them both into consideration. Firstly, we propose a new multi-scale matching cost computation sub-network, in which two different sizes of receptive fields are implemented parallelly. In this way, the network can make the best use of both variants to balance the trade-off between the increase of receptive field and the loss of details. Furthermore, we show that our multi-dimension aggregation sub-network which contains 2D convolution and 3D convolution operations can provide rich context and semantic information for estimating an accurate initial disparity. Finally, experiments on challenging stereo benchmark KITTI demonstrate that the proposed method can achieve competitive results even without any additional post-processing.

A Greedy Approach for a Time-Dependent Scheduling Problem

Disparity Estimation Using Multilevel and Global Information

Efficient Binocular Stereo Matching Based on Sad and Improved Census Transformation

Modeling Stereo-Confidence Out of the End-to-End Stereo-Matching Network via Disparity Plane Sweep

Multi-Dimensional Cooperative Network for Stereo Matching

Stereo Matching Using Multi-Level Cost Volume and Multi-Scale Feature Constancy

A Joint 2D-3D Complementary Network for Stereo Matching

Deep Stereo Matching With Hysteresis Attention and Supervised Cost Volume Construction

An efficient and accurate multi-level cascaded recurrent network for stereo matching

A Fast Stereo Matching Network with Multi-Cross Attention

HLocalExp-CM: confidence map by hierarchical local expansion moves for accurate stereo matching

Practical Stereo Matching via Cascaded Recurrent Network with Adaptive Correlation

Temporally Consistent Stereo Matching

End-to-End Edge-Guided Multi-Scale Matching Network for Optical Satellite Stereo Image Pairs

Stereo Matching by Self-supervision of Multiscopic Vision.

End-to-End Learning of Multi-scale Convolutional Neural Network for Stereo Matching

Parallax attention stereo matching network based on the improved group-wise correlation stereo network

EAI-Stereo: Error Aware Iterative Network for Stereo Matching

Multi-Scale Cost Volumes Cascade Network for Stereo Matching

Multi-path Feature Mining Network for Stereo Matching

Cascaded multi-scale and multi-dimension convolutional neural network for stereo matching.