Abstract:Person re-identification (Re-ID) is an important problem in video surveillance for matching pedestrian images across non-overlapping camera views. Currently, most works focus on RGB-based Re-ID. However, RGB images are not well suited to a dark environment; consequently, infrared (IR) imaging becomes necessary for indoor scenes with low lighting and 24-h outdoor scene surveillance systems. In such scenarios, matching needs to be performed between RGB images and IR images, which exhibit different visual characteristics; this cross-modality matching problem is more challenging than RGB-based Re-ID due to the lack of visible colour information in IR images. To address this challenge, we study the RGB-IR cross-modality Re-ID (RGB-IR Re-ID) problem. Rather than applying existing cross-modality matching models that operate under the assumption of identical data distributions between training and testing sets to handle the discrepancy between RGB and IR modalities for Re-ID, we cast learning shared knowledge for cross-modality matching as the problem of cross-modality similarity preservation. We exploit same-modality similarity as the constraint to guide the learning of cross-modality similarity along with the alleviation of modality-specific information, and finally propose a Focal Modality-Aware Similarity-Preserving Loss. To further assist the feature extractor in extracting shared knowledge, we design a modality-gated node as a universal representation of both modality-specific and shared structures for constructing a structure-learnable feature extractor called Modality-Gated Extractor. For validation, we construct a new multi-modality Re-ID dataset, called SYSU-MM01, to enable wider study of this problem. Extensive experiments on this SYSU-MM01 dataset show the effectiveness of our method. Download link of dataset: <a href="https://github.com/wuancong/SYSU-MM01">https://github.com/wuancong/SYSU-MM01</a>.

ARCTIC: A knowledge distillation approach via attention-based relation matching and activation region constraint for RGB-to-Infrared videos action recognition

Modality-transfer Generative Adversarial Network and Dual-Level Unified Latent Representation for Visible Thermal Person Re-Identification

A Novel 3D Convolutional Neural Network for Action Recognition in Infrared Videos

A New Dataset and Evaluation for Infrared Action Recognition.

Learning and Distillating the Internal Relationship of Motion Features in Action Recognition.

Human Action Recognition with Contextual Constraints Using a RGB-D Sensor

Cross-modality Online Distillation for Multi-View Action Recognition

InfAR Dataset: Infrared Action Recognition at Different Times

Modality-Adaptive Mixup and Invariant Decomposition for RGB-Infrared Person Re-Identification

Transferable Feature Representation for Visible-to-Infrared Cross-Dataset Human Action Recognition

Learning an Augmented RGB Representation with Cross-Modal Knowledge Distillation for Action Detection

Learning Spatiotemporal Features for Infrared Action Recognition with 3D Convolutional Neural Networks

Action Recognition in Depth Video from RGB Perspective: A Knowledge Transfer Manner

RGB-IR Person Re-identification by Cross-Modality Similarity Preservation

Fine-grained action recognition using multi-view attentions

Global Temporal Representation based CNNs for Infrared Action Recognition

Infrared and 3D skeleton feature fusion for RGB-D action recognition

Temporal Cues Enhanced Multimodal Learning for Action Recognition in RGB-D Videos

Robust action recognition via borrowing information across video modalities

Two-stream Siamese Network with Contrastive-Center Losses for RGB-D Action Recognition

Trear: Transformer-Based RGB-D Egocentric Action Recognition