Abstract:RGBT tracking is receiving more and more attention because of its huge tracking potential in an all-weather environment. RGB and thermal source data contain different levels of information about the object. Utilizing the complementary advantage of different levels of information can effectively improve the tracking performance. Existing work focuses on the extraction and fusion of multi-modal features. Although these methods effectively deploy the fusion of information among multiple modalities, they ignore the potential value of multi-level shared clues in different modalities. In addition, these works cannot provide effective candidate boxes after tracking drift, resulting in limited tracker performance. In this paper, we propose a cross-modality interaction and re-identification network that performs multi-level modality-shared, modality-specific and object probability prediction learning. We designed two feature extraction sub-networks, namely, a multi-level modality-shared fusion network and modality complementary sub-network. Specifically, the two sub-networks extract and fuse multi-level modality shared information and modality specific information, respectively. To optimize tracking drift, object-aware branches that predict the object-centered state are designed. Our object-aware branching is simple, neat and efficient. Moreover, to achieve the visual tracking real-time requirement, we designed the object regression branch that does not require repeated region suggestion input. By extensive experiments and comparisons with state-of-the-art trackers on the RGBT tracking benchmark dataset, our tracker achieves leading performance and essentially real-time tracking speeds. Tracking drift caused by occlusion, fast motion and camera moving is significantly optimized.

Simplifying Cross-modal Interaction Via Modality-Shared Features for RGBT Tracking

Unidirectional Cross-Modal Fusion for RGB-T Tracking

CIRNet: an Improved RGBT Tracking Via Cross-Modality Interaction and Re-Identification

RGBT Tracking Based on Modality Feature Enhancement

Multi-modal interaction with token division strategy for RGB-T tracking

RGB-T Tracking with Template-Bridged Search Interaction and Target-Preserved Template Updating

RGB-T tracking by modality difference reduction and feature re-selection

Special attribute-based cross-modal interactive fusion network for RGBT tracking

Robust RGB-T Tracking Via Adaptive Modality Weight Correlation Filters and Cross-modality Learning

Towards Modalities Correlation for RGB-T Tracking

Object fusion tracking for RGB-T images via channel swapping and modal mutual attention

Learning Modality Complementary Features with Mixed Attention Mechanism for RGB-T Tracking

Cross-Modal Object Tracking: Modality-Aware Representations and a Unified Benchmark

Visible and Infrared Object Tracking Based on Multimodal Hierarchical Relationship Modeling

Cross-modal Attention Network for RGB-T Tracking

RGB-T Tracking Based on Mixed Attention

Multi-Level Fusion for Robust RGBT Tracking via Enhanced Thermal Representation

Temporal Adaptive RGBT Tracking with Modality Prompt

RGBT tracking via cross-modality message passing

Asymmetric Global–Local Mutual Integration Network for RGBT Tracking

RGBT Tracking via Challenge-Based Appearance Disentanglement and Interaction