Abstract:Visual tracking has made significant strides due to the adoption of transformer-based models. Most state-of-the-art trackers struggle to meet real-time processing demands on mobile platforms with constrained computing resources, particularly for real-time unmanned aerial vehicle (UAV) tracking. To achieve a better balance between performance and efficiency, we introduce AVTrack, an adaptive computation framework designed to selectively activate transformer blocks for real-time UAV tracking. The proposed Activation Module (AM) dynamically optimizes the ViT architecture by selectively engaging relevant components, thereby enhancing inference efficiency without significant compromise to tracking performance. Furthermore, to tackle the challenges posed by extreme changes in viewing angles often encountered in UAV tracking, the proposed method enhances ViTs' effectiveness by learning view-invariant representations through mutual information (MI) maximization. Two effective design principles are proposed in the AVTrack. Building on it, we propose an improved tracker, dubbed AVTrack-MD, which introduces the novel MI maximization-based multi-teacher knowledge distillation (MD) framework. It harnesses the benefits of multiple teachers, specifically the off-the-shelf tracking models from the AVTrack, by integrating and refining their outputs, thereby guiding the learning process of the compact student network. Specifically, we maximize the MI between the softened feature representations from the multi-teacher models and the student model, leading to improved generalization and performance of the student model, particularly in noisy conditions. Extensive experiments on multiple UAV tracking benchmarks demonstrate that AVTrack-MD not only achieves performance comparable to the AVTrack baseline but also reduces model complexity, resulting in a significant 17\% increase in average tracking speed.

Mobile Vision Transformer-based Visual Object Tracking

AViTMP: A Tracking-Specific Transformer for Single-Branch Visual Tracking

Exploring Lightweight Hierarchical Vision Transformers for Efficient Visual Tracking

Compact Transformer Tracker with Correlative Masked Modeling.

ViTT: Vision Transformer Tracker

Adaptively Bypassing Vision Transformer Blocks for Efficient Visual Tracking

VTT: Long-term Visual Tracking with Transformers

MobileViT: Light-weight, General-purpose, and Mobile-friendly Vision Transformer

DMTrack: learning deformable masked visual representations for single object tracking

GTT: Visual Tracking with Gaussion Transformer

GTTrack: Gaussian Transformer Tracker for Visual Tracking.

Heterogeneous Graph Transformer for Multiple Tiny Object Tracking in RGB-T Videos

CVTrack: Combined Convolutional Neural Network and Vision Transformer Fusion Model for Visual Tracking

MTAtrack: Multilevel Transformer Attention for Visual Tracking

MIMTracking: Masked image modeling enhanced vision transformer for visual object tracking

PromptVT: Prompting for Efficient and Accurate Visual Tracking

Global Tracking Transformers

Adaptive and Background-Aware Vision Transformer for Real-Time UAV Tracking

Learning Adaptive and View-Invariant Vision Transformer with Multi-Teacher Knowledge Distillation for Real-Time UAV Tracking

Target-aware transformer tracking with hard occlusion instance generation

MambaEVT: Event Stream based Visual Object Tracking using State Space Model