Abstract:RGB-Thermal (RGB-T) pedestrian detection aims to locate pedestrians in RGB-T image pairs to exploit the complementation between the two modalities for improving detection robustness in extreme conditions. Most existing algorithms assume that the RGB-T image pairs are well registered, while in the real world, they are not ideally aligned due to parallax or different field-of-view of the cameras. The pedestrians in misaligned image pairs may be located at different positions in two images, which results in two challenges: 1) how to achieve inter-modality complementation using spatially misaligned RGB-T pedestrian patches and 2) how to recognize unpaired pedestrians at the boundary. To address these issues, we propose a new paradigm for unregistered RGB-T pedestrian detection, which predicts two separate pedestrian locations in RGB and thermal images. Specifically, we propose a cross-modality proposal-guided feature mining (CPFM) mechanism to extract two precise fusion features for representing a pedestrian in the two modalities, even if the given RGB-T image pair is unaligned. It enables us to effectively exploit the complementation between the two modalities. With the CPFM mechanism, we build a two-stream dense detector that predicts two pedestrian locations in the two modalities based on the corresponding fusion features mined by the CPFM mechanism. In addition, we design a data augmentation method, named Homography, to simulate the discrepancy in scales and views between images. We also investigate two non-maximum suppression (NMS) methods for post-processing purposes. Favorable experimental results demonstrate the effectiveness and robustness of our method in addressing unregistered pedestrians with different shifts.

A Hybrid Approach to Pedestrian Clothing Color Attribute Extraction

Pedestrian Attribute Recognition Via Spatio-temporal Relationship Learning for Visual Surveillance

Dual-branch Self-Attention Network for Pedestrian Attribute Recognition

An efficient pedestrian attributes recognition system under challenging conditions

Learning Cross-Modal Deep Representations for Robust Pedestrian Detection

Locality guided cross-modal feature aggregation and pixel-level fusion for multispectral pedestrian detection

A Pedestrian-Detection Method Based on Heterogeneous Features and Ensemble of Multi-View–Pose Parts

Pedestrian attribute recognition: A survey

Cross-Modality Proposal-guided Feature Mining for Unregistered RGB-Thermal Pedestrian Detection

Learning Transferable Pedestrian Representation from Multimodal Information Supervision

Weight Determination In Multi-Feature Fusion For Pedestrian Re-Identification

Pedestrian Attribute Recognition: A New Benchmark Dataset and A Large Language Model Augmented Framework

Localization Guided Learning for Pedestrian Attribute Recognition

Multiscale Cross-modal Homogeneity Enhancement and Confidence-aware Fusion for Multispectral Pedestrian Detection

Evaluation of local feature descriptors and their combination for pedestrian representation

Pedestrian Detection Using Hybrid Statistical Feature.

Deep Template Matching for Pedestrian Attribute Recognition with the Auxiliary Supervision of Attribute-wise Keypoints

An Approach for Clothing Recommendation Based on Multiple Image Attributes

Bright Channel Prior Attention for Multispectral Pedestrian Detection

Hybrid Attention for Robust RGB-T Pedestrian Detection in Real-World Conditions