Omnidirectional Depth Estimation With Hierarchical Deep Network for Multi-Fisheye Navigation Systems
Xiaojie Su,Shimin Liu,Rui Li
DOI: https://doi.org/10.1109/tits.2023.3294642
IF: 8.5
2023-01-01
IEEE Transactions on Intelligent Transportation Systems
Abstract:Multi-fisheye System has the advantages of sufficient overlap and the ability to capture a complete 360 $^{\circ}$ scene, which is beneficial for the omnidirectional depth estimation task. However, due to the severe distortion of the fisheye images, it is hard for such systems to extract and match features to predict an accurate depth. In this work, on the basis of a multi-fisheye system, we present a novel end-to-end deep learning architecture for omnidirectional depth estimation: 1) to capture the reliable features of the distorted fisheye image, a multi-scale feature extraction and aggregation module is improved, which can adaptively obtain the global context information to represent the features; 2) to leverage more aligned features, especially those in the overlap between multi-fisheye images, we construct a fusion cost volume to combine similarity and semantic information, which can enhance the feature discriminability; and 3) to refine the omnidirectional depth map efficiently, a cascaded cost regularization architecture is proposed. Instead of several costly 3D convolutions, the 3D BSConv based on intra-kernel correlations is introduced to regularize the cost. The proposed method can incrementally predict the depth map from coarse to fine, and reduce the network computational complexity significantly. The experiments in several public indoor and outdoor synthetic datasets demonstrate that the proposed method outperforms some state-of-the-art methods in terms of a synthesis of accuracy and speed, with fewer model parameters.
engineering, electrical & electronic,transportation science & technology, civil