Abstract:RGB-Thermal (RGB-T) pedestrian detection aims to locate pedestrians in RGB-T image pairs to exploit the complementation between the two modalities for improving detection robustness in extreme conditions. Most existing algorithms assume that the RGB-T image pairs are well registered, while in the real world, they are not ideally aligned due to parallax or different field-of-view of the cameras. The pedestrians in misaligned image pairs may be located at different positions in two images, which results in two challenges: 1) how to achieve inter-modality complementation using spatially misaligned RGB-T pedestrian patches and 2) how to recognize unpaired pedestrians at the boundary. To address these issues, we propose a new paradigm for unregistered RGB-T pedestrian detection, which predicts two separate pedestrian locations in RGB and thermal images. Specifically, we propose a cross-modality proposal-guided feature mining (CPFM) mechanism to extract two precise fusion features for representing a pedestrian in the two modalities, even if the given RGB-T image pair is unaligned. It enables us to effectively exploit the complementation between the two modalities. With the CPFM mechanism, we build a two-stream dense detector that predicts two pedestrian locations in the two modalities based on the corresponding fusion features mined by the CPFM mechanism. In addition, we design a data augmentation method, named Homography, to simulate the discrepancy in scales and views between images. We also investigate two non-maximum suppression (NMS) methods for post-processing purposes. Favorable experimental results demonstrate the effectiveness and robustness of our method in addressing unregistered pedestrians with different shifts.

Boosting Associated Pairing Comparison Features for Pedestrian Detection.

Local Associated Features For Pedestrian Detection

Locality guided cross-modal feature aggregation and pixel-level fusion for multispectral pedestrian detection

Multi Features Combination for Pedestrian Detection.

Pair Normalized Channel Feature and Statistics-Based Learning for High-Performance Pedestrian Detection

Center-Surround Contrast Features for Pedestrian Detection

Cross-Modality Proposal-guided Feature Mining for Unregistered RGB-Thermal Pedestrian Detection

Matching of Tracked Pedestrians Across Disjoint Camera Views Using CI-DLBP.

Exploring Human Vision Driven Features for Pedestrian Detection.

The Combination of CSLBP and LBP Feature for Pedestrian Detection

A robust pedestrian detector based on heterogeneous feature fusion

A Pedestrian-Detection Method Based on Heterogeneous Features and Ensemble of Multi-View–Pose Parts

Pedestrian Proposal and Refining Based on the Shared Pixel Differential Feature

CAPNet: Context and Attribute Perception for Pedestrian Detection

Local Co-Occurrence Selection Via Partial Least Squares for Pedestrian Detection

Oriented Gradient Context for Pedestrian Detection

Fast Pedestrian Detection Using a Cascade of Boosted Covariance Features.

A Boosted Multi-Task Model for Pedestrian Detection with Occlusion Handling.

A New Pedestrian Detection Method Based on Combined HOG and LSS Features

Color Self-Similarity Feature Based Real-Time Pedestrian Detection