Abstract:Pedestrian detection plays a critical role in computer vision as it contributes to ensuring traffic safety. Existing methods that rely solely on RGB images suffer from performance degradation under low-light conditions due to the lack of useful information. To address this issue, recent multispectral detection approaches have combined thermal images to provide complementary information and have obtained enhanced performances. Nevertheless, few approaches focus on the negative effects of false positives (FPs) caused by noisy fused feature maps. Different from them, we comprehensively analyze the impacts of FPs on detection performance and find that enhancing feature contrast can significantly reduce these FPs. In this article, we propose a novel target-aware fusion strategy for multispectral pedestrian detection, named TFDet. The target-aware fusion strategy employs a fusion-refinement paradigm. In the fusion phase, we reveal the parallel-and cross-channel similarities in RGB and thermal features and learn an adaptive receptive field to collect useful information from both features. In the refinement phase, we use a segmentation branch to discriminate the pedestrian features from the background features. We propose a correlation-maximum loss function to enhance the contrast between the pedestrian features and background features. As a result, our fusion strategy highlights pedestrian-related features and suppresses unrelated ones, generating more discriminative fused features. TFDet achieves state-of-the-art performance on two multispectral pedestrian benchmarks, KAIST and LLVIP, with absolute gains of 0.65% and 4.1% over the previous best approaches, respectively. TFDet can easily extend to multiclass object detection scenarios. It outperforms the previous best approaches on two multispectral object detection benchmarks, FLIR and M3FD, with absolute gains of 2.2% and 1.9%, respectively. Importantly, TFDet has comparable inference efficiency to the previous approaches and has remarkably good detection performance even under low-light conditions, which is a significant advancement for ensuring road safety. The code will be made publicly available at https://github.com/XueZ-phd/TFDet.git.

Fused Discriminative Metric Learning for Low Resolution Pedestrian Detection.

Online Multipedestrian Tracking Based on Fused Detections of Millimeter Wave Radar and Vision

Locality guided cross-modal feature aggregation and pixel-level fusion for multispectral pedestrian detection

Transformer fusion and histogram layer multispectral pedestrian detection network

Discriminative Latent Semantic Feature Learning for Pedestrian Detection

Pedestrian Detection Using Multi-Channel Visual Feature Fusion by Learning Deep Quality Model.

A Pedestrian Detection Algorithm Based on Score Fusion for Multi-LiDAR Systems

TFDet: Target-Aware Fusion for RGB-T Pedestrian Detection

Local Fisher Discriminant Analysis for Pedestrian Re-identification

FCL: Pedestrian Re-Identification Algorithm Based on Feature Fusion Contrastive Learning

Multi-Grained Deep Feature Learning for Pedestrian Detection

Fused DNN: A deep neural network fusion approach to fast and robust pedestrian detection

Occluded Pedestrian Detection via Distribution-Based Mutual-Supervised Feature Learning

Deep Feature Fusion by Competitive Attention for Pedestrian Detection

Weighted-Diffusion Distance Measurement Ranking for Pedestrian Re-Identification

Pedestrian Re-Identification Based on Fine-Grained Feature Learning and Fusion

Cross-Modality Proposal-guided Feature Mining for Unregistered RGB-Thermal Pedestrian Detection

Posterior Probability Based Multi-classifier Fusion in Pedestrian Detection.