Multispectral pedestrian detection based on feature complementation and enhancement
Linzhen Nie,Meihe Lu,Zhiwei He,Jiachen Hu,Zhishuai Yin
DOI: https://doi.org/10.1049/itr2.12562
IF: 2.7
2024-10-19
IET Intelligent Transport Systems
Abstract:A novel multispectral pedestrian detector is proposed and validated on the KAIST benchmark and CVC‐14 dataset, showing competitive all‐day performance and superiority in nighttime detection, with a competitive inference time. Multispectral pedestrian detection with visible light and infrared images is robust to changes in lighting conditions and therefore is of great importance to numerous applications that require all‐day environmental perception. This paper proposes a novel method named FCE‐RCNN, which integrates saliency detection as a sub‐task and utilizes global information for enhanced feature representation. The approach enhances thermal inputs by incorporating gradients at the raw‐data level before feature extraction. Utilizing a dual‐stream backbone, a global semantic information extraction module is introduced that combines pooling with horizontal–vertical attention mechanisms, capturing high‐quality global semantic information for lower‐level feature enrichment and guidance. Additionally, the pedestrian locality enhancement module is designed to enhance spatial locality information of pedestrians through saliency detection. Furthermore, to alleviate the challenges posed by positional shifts between cross‐spectral features, deformable convolution is innovatively employed. Experimental results on the KAIST dataset demonstrate that FCE‐RCNN significantly improves nighttime detection, achieving a log‐average miss rate of 6.92%, outperforming the new method ICAFusion by 0.93%. These results underscore the effectiveness of FCE‐RCNN, and the method also maintains competitive inference speed, making it suitable for real‐time applications.
engineering, electrical & electronic,transportation science & technology