Dilated High-Resolution Network Driven RGB-T Multi-Modal Crowd Counting

Zhengyi Liu,Yacheng Tan,Wei Wu,Bin Tang
DOI: https://doi.org/10.1016/j.image.2022.116915
IF: 3.453
2023-01-01
Signal Processing Image Communication
Abstract:Crowd counting aims to estimate the number of pedestrians in a scene. However, the problems of insufficient illumination and large-scale variation affect the accuracy of crowd counting. In this paper, a dilated high-resolution network (DHRNet) driven RGB-T multi-modal crowd counting model is proposed to address the above problems. In terms of the importance of RGB and thermal modalities, a thermal-main and RGB-auxiliary strategy is chosen instead of treating them equally as in previous works. In terms of the fusion of RGB and thermal modalities, a cross-modal fusion module is designed and embedded in the input feature level of DHRNet. In terms of DHRNet output feature utilization, a multilayer perceptron regression head is proposed to predict high-quality density maps. The experimental results on public datasets show that our proposed network significantly outperforms state-of-the-art methods and reaches a new level of performance. The ablation studies verify the effectiveness of the thermal-main and RGB-auxiliary strategy and the proposed modules.
What problem does this paper attempt to address?