Abstract:RGB-Thermal (RGB-T) pedestrian detection aims to locate the pedestrians in RGB-T image pairs to exploit the complementation between the two modalities for improving detection robustness in extreme conditions. Most existing algorithms assume that the RGB-T image pairs are well registered, while in the real world they are not aligned ideally due to parallax or different field-of-view of the cameras. The pedestrians in misaligned image pairs may locate at different positions in two images, which results in two challenges: 1) how to achieve inter-modality complementation using spatially misaligned RGB-T pedestrian patches, and 2) how to recognize the unpaired pedestrians at the boundary. To deal with these issues, we propose a new paradigm for unregistered RGB-T pedestrian detection, which predicts two separate pedestrian locations in the RGB and thermal images, respectively. Specifically, we propose a cross-modality proposal-guided feature mining (CPFM) mechanism to extract the two precise fusion features for representing the pedestrian in the two modalities, even if the RGB-T image pair is unaligned. It enables us to effectively exploit the complementation between the two modalities. With the CPFM mechanism, we build a two-stream dense detector; it predicts the two pedestrian locations in the two modalities based on the corresponding fusion feature mined by the CPFM mechanism. Besides, we design a data augmentation method, named Homography, to simulate the discrepancy in scales and views between images. We also investigate two non-maximum suppression (NMS) methods for post-processing. Favorable experimental results demonstrate the effectiveness and robustness of our method in dealing with unregistered pedestrians with different shifts.

Spatial-Temporal Image Completion via Global Optimization for Pedestrian Removal

Towards Accurate Dense Pedestrian Detection Via Occlusion-Prediction Aware Label Assignment and Hierarchical-Nms.

A Global Space-Time Optimization Framework for Video Completion

Locality guided cross-modal feature aggregation and pixel-level fusion for multispectral pedestrian detection

Automatic Objects Removal for Scene Completion

Crosswalk Detection from Satellite Imagery for Pedestrian Network Completion

Structure maps based pedestrian detection

Exploiting the Potential of Overlapping Cropping for Real-World Pedestrian and Vehicle Detection with Gigapixel-Level Images

Contour-motion Feature (CMF): A Space-Time Approach for Robust Pedestrian Detection

Context feature fusion and enhanced non-maximum suppression for pedestrian detection in crowded scenes

Cross-Modality Proposal-guided Feature Mining for Unregistered RGB-Thermal Pedestrian Detection

MambaST: A Plug-and-Play Cross-Spectral Spatial-Temporal Fuser for Efficient Pedestrian Detection

Unsupervised Multi-view Pedestrian Detection

Multi-Scale Structure Perception and Global Context-Aware Method for Small-Scale Pedestrian Detection

Restoration of fragmentary trajectories using evaluation of identity and combinatorial optimization

Video Completion Using Global Optimization

Image completion based on views of large displacement

AutoPedestrian: An Automatic Data Augmentation and Loss Function Search Scheme for Pedestrian Detection

Taking a Look at Small-Scale Pedestrians and Occluded Pedestrians

Dynamic Region Division for Adaptive Learning Pedestrian Counting