Abstract:Multispectral pedestrian detection is a technology designed to detect and locate pedestrians in Color and Thermal images, which has been widely used in automatic driving, video surveillance, etc. So far most available multispectral pedestrian detection algorithms only achieved limited success in pedestrian detection because of the lacking take into account the confusion of pedestrian information and background noise in Color and Thermal images. Here we propose a multispectral pedestrian detection algorithm, which mainly consists of a cascaded information enhancement module and a cross-modal attention feature fusion module. On the one hand, the cascaded information enhancement module adopts the channel and spatial attention mechanism to perform attention weighting on the features fused by the cascaded feature fusion block. Moreover, it multiplies the single-modal features with the attention weight element by element to enhance the pedestrian features in the single-modal and thus suppress the interference from the background. On the other hand, the cross-modal attention feature fusion module mines the features of both Color and Thermal modalities to complement each other, then the global features are constructed by adding the cross-modal complemented features element by element, which are attentionally weighted to achieve the effective fusion of the two modal features. Finally, the fused features are input into the detection head to detect and locate pedestrians. Extensive experiments have been performed on two improved versions of annotations (sanitized annotations and paired annotations) of the public dataset KAIST. The experimental results show that our method demonstrates a lower pedestrian miss rate and more accurate pedestrian detection boxes compared to the comparison method. Additionally, the ablation experiment also proved the effectiveness of each module designed in this paper.

MGA-YOLOv4: a multi-scale pedestrian detection method based on mask-guided attention

Pedestrian Detection Method Based on Improved YOLOv5s for Densely Occluded Scenarios

Towards Accurate Dense Pedestrian Detection Via Occlusion-Prediction Aware Label Assignment and Hierarchical-Nms.

MAF-YOLO: Multi-modal attention fusion based YOLO for pedestrian detection

A Pedestrian Target Detection Network Based on Attention Mechanism and Multi-scale Feature Fusion

Pedestrian Partial Occlusion Detection Method Based on Improved YOLO Network Structure

A Multiscale Parallel Pedestrian Recognition Algorithm Based on YOLOv5

Mask-Guided Attention Network for Occluded Pedestrian Detection

YOLOv8-based Dense Pedestrian Detection Algorithm

Locality guided cross-modal feature aggregation and pixel-level fusion for multispectral pedestrian detection

A Lightweight Vehicle-Pedestrian Detection Algorithm Based on Attention Mechanism in Traffic Scenarios

Little-YOLOv4: A Lightweight Pedestrian Detection Network Based on YOLOv4 and GhostNet

SE-YOLOv4: shuffle expansion YOLOv4 for pedestrian detection based on PixelShuffle

A Pedestrian Detection Network Model Based on Improved YOLOv5

Reparameterized dilated architecture: A wider field of view for pedestrian detection

An Improved Pedestrian Detection Model Based on YOLOv8 for Dense Scenes

LGADet: Light-weight Anchor-free Multispectral Pedestrian Detection with Mixed Local and Global Attention

Multi-Scale Structure Perception and Global Context-Aware Method for Small-Scale Pedestrian Detection

YOLO-ABD: A Multi-Scale Detection Model for Pedestrian Anomaly Behavior Detection

Cascaded information enhancement and cross-modal attention feature fusion for multispectral pedestrian detection

YOLOv8-CB: Dense Pedestrian Detection Algorithm Based on In-Vehicle Camera