Abstract:A novel multispectral pedestrian detector is proposed and validated on the KAIST benchmark and CVC‐14 dataset, showing competitive all‐day performance and superiority in nighttime detection, with a competitive inference time. Multispectral pedestrian detection with visible light and infrared images is robust to changes in lighting conditions and therefore is of great importance to numerous applications that require all‐day environmental perception. This paper proposes a novel method named FCE‐RCNN, which integrates saliency detection as a sub‐task and utilizes global information for enhanced feature representation. The approach enhances thermal inputs by incorporating gradients at the raw‐data level before feature extraction. Utilizing a dual‐stream backbone, a global semantic information extraction module is introduced that combines pooling with horizontal–vertical attention mechanisms, capturing high‐quality global semantic information for lower‐level feature enrichment and guidance. Additionally, the pedestrian locality enhancement module is designed to enhance spatial locality information of pedestrians through saliency detection. Furthermore, to alleviate the challenges posed by positional shifts between cross‐spectral features, deformable convolution is innovatively employed. Experimental results on the KAIST dataset demonstrate that FCE‐RCNN significantly improves nighttime detection, achieving a log‐average miss rate of 6.92%, outperforming the new method ICAFusion by 0.93%. These results underscore the effectiveness of FCE‐RCNN, and the method also maintains competitive inference speed, making it suitable for real‐time applications.

A Fast RetinaNet Fusion Framework for Multi-Spectral Pedestrian Detection

Multispectral Deep Neural Networks for Pedestrian Detection

Transformer fusion and histogram layer multispectral pedestrian detection network

MRF3Net: Infrared Small Target Detection Using Multi-Receptive Field Perception and Effective Feature Fusion

Robust Pedestrian Detection Based on Multi-Spectral Image Fusion and Convolutional Neural Networks

Optimal Fusion-based Asymmetric Two-stream Networks for Multispectral Image Pedestrian Detection

Multispectral pedestrian detection based on feature complementation and enhancement

Fused DNN: A deep neural network fusion approach to fast and robust pedestrian detection

Illumination-aware Faster R-CNN for Robust Multispectral Pedestrian Detection

Fusion of Multispectral Data Through Illumination-aware Deep Neural Networks for Pedestrian Detection

Box-level Segmentation Supervised Deep Neural Networks for Accurate and Real-time Multispectral Pedestrian Detection

Exploiting fusion architectures for multispectral pedestrian detection and segmentation

Deep saliency detection-based pedestrian detection with multispectral multi-scale features fusion network

Pedestrian detection-driven cascade network for infrared and visible image fusion

Multispectral Pedestrian Detection via Simultaneous Detection and Segmentation

TFDet: Target-Aware Fusion for RGB-T Pedestrian Detection

Pedestrian detection with unsupervised multispectral feature learning using deep neural networks

Locality guided cross-modal feature aggregation and pixel-level fusion for multispectral pedestrian detection

INSANet: INtra-INter Spectral Attention Network for Effective Feature Fusion of Multispectral Pedestrian Detection

TS-Net: two-stream network based on multispectral feature fusion