Abstract:Many RGBT tracking researches primarily focus on modal fusion design, while overlooking the effective handling of target appearance changes. While some approaches have introduced historical frames or fuse and replace initial templates to incorporate temporal information, they have the risk of disrupting the original target appearance and accumulating errors over time. To alleviate these limitations, we propose a novel Transformer RGBT tracking approach, which mixes spatio-temporal multimodal tokens from the static multimodal templates and multimodal search regions in Transformer to handle target appearance changes, for robust RGBT tracking. We introduce independent dynamic template tokens to interact with the search region, embedding temporal information to address appearance changes, while also retaining the involvement of the initial static template tokens in the joint feature extraction process to ensure the preservation of the original reliable target appearance information that prevent deviations from the target appearance caused by traditional temporal updates. We also use attention mechanisms to enhance the target features of multimodal template tokens by incorporating supplementary modal cues, and make the multimodal search region tokens interact with multimodal dynamic template tokens via attention mechanisms, which facilitates the conveyance of multimodal-enhanced target change information. Our module is inserted into the transformer backbone network and inherits joint feature extraction, search-template matching, and cross-modal interaction. Extensive experiments on three RGBT benchmark datasets show that the proposed approach maintains competitive performance compared to other state-of-the-art tracking algorithms while running at 39.1 FPS.

Effective and Robust: A Discriminative Temporal Learning Transformer for Satellite Videos

Adaptive Multi-Scale Transformer Tracker for Satellite Videos

Exploiting Temporal Coherence for Self-Supervised Visual Tracking by Using Vision Transformer

RTSformer: A Robust Toroidal Transformer With Spatiotemporal Features for Visual Tracking

DTA: Deformable Temporal Attention for Video Recognition

VideoTrack: Learning to Track Objects Via Video Transformer

Temporal Relation Transformer for Robust Visual Tracking with Dual-Memory Learning

Transformer Meets Tracker: Exploiting Temporal Context for Robust Visual Tracking

Target-Aware Transformer for Satellite Video Object Tracking

Transformer Tracking for Satellite Video: Matching, Propagation, and Prediction

A Transformer-based Visual Object Tracker Via Learning Immediate Appearance Change

Towards Robust Video Instance Segmentation with Temporal-Aware Transformer

Exploring reliable infrared object tracking with spatio-temporal fusion transformer

Transformer RGBT Tracking with Spatio-Temporal Multimodal Tokens

High-Precision Multi-Object Tracking in Satellite Videos via Pixel-Wise Adaptive Feature Enhancement

Exploring Dynamic Transformer for Efficient Object Tracking

Adaptive sparse attention-based compact transformer for object tracking

Discriminative Target Predictor Based on Temporal-Scene Attention Context Enhancement and Candidate Matching Mechanism

Efficient Video Transformers with Spatial-Temporal Token Selection

Region-Aware Temporal Inconsistency Learning for DeepFake Video Detection

TFITrack: Transformer Feature Integration Network for Object Tracking