Agspn: Efficient Attention-Gated Spatial Propagation Network for Depth Completion
Huajie Wen,Jianhao Shen,Cheng Chi,Lin,Qiaohui Feng,Gang Xu
DOI: https://doi.org/10.1016/j.eswa.2024.125939
IF: 8.5
2024-01-01
Expert Systems with Applications
Abstract:Sparse depth maps and their corresponding RGB images can be used to restore the dense depth map, which is the main purpose of depth completion and plays an important role in the field of autonomous driving and scene awareness. The existing spatial propagation networks have been widely used in depth completion, but their fixed affinity representation is limited during the iteration process, and the large computational load is also needed to ensure acceptable accuracy. And therefore, the corresponding algorithms could not be running on such platforms as performance-constrained embedded computers. To address these problems, a lightweight attention-gated spatial propagation network is proposed to complete sparse depth maps. Firstly, RGB images are fused with sparse LiDAR images through a backbone network, and then rough initial depth, diffusion offset, affinity, and feature maps are predicted. Subsequently, multiple affinity matrices are computed at different stages, and the initial depth is dynamically iteratively propagated to enhance the affinity representation capability. Finally, the attention and gating mechanism are used to calculate the weight coefficients for different stages by multiplying the coefficients with the depth maps of different stages and summing them up to obtain the final depth map. The proposed method has achieved excellent or competitive results in extensive experiments on the KITTI depth completion benchmark and NYU-depth-V2, with the least number of floating-point operations per second. Importantly, it can be deployed on embedded platforms to infer in real-time and balance between errors and computational requirement.