Abstract:Developing robust and discriminative appearance models has been a long-standing research challenge in visual object tracking. In the prevalent Siamese-based paradigm, the features extracted by the Siamese-like networks are often insufficient to model the tracked targets and distractor objects, thereby hindering them from being robust and discriminative simultaneously. While most Siamese trackers focus on designing robust correlation operations, we propose a novel single-branch tracking framework inspired by the transformer. Unlike the Siamese-like feature extraction, our tracker deeply embeds cross-image feature correlation in multiple layers of the feature network. By extensively matching the features of the two images through multiple layers, it can suppress non-target features, resulting in target-aware feature extraction. The output features can be directly used for predicting target locations without additional correlation steps. Thus, we reformulate the two-branch Siamese tracking as a conceptually simple, fully transformer-based Single-Branch Tracking pipeline, dubbed SBT. After conducting an in-depth analysis of the SBT baseline, we summarize many effective design principles and propose an improved tracker dubbed SuperSBT. SuperSBT adopts a hierarchical architecture with a local modeling layer to enhance shallow-level features. A unified relation modeling is proposed to remove complex handcrafted layer pattern designs. SuperSBT is further improved by masked image modeling pre-training, integrating temporal modeling, and equipping with dedicated prediction heads. Thus, SuperSBT outperforms the SBT baseline by 4.7%,3.0%, and 4.5% AUC scores in LaSOT, TrackingNet, and GOT-10K. Notably, SuperSBT greatly raises the speed of SBT from 37 FPS to 81 FPS. Extensive experiments show that our method achieves superior results on eight VOT benchmarks.

CRTrack: Learning Correlation-Refine network for visual object tracking

CFNN: Correlation Filter Neural Network for Visual Object Tracking

Deep Correlation Filter based Real-Time Tracker

Correlation-Aware Deep Tracking

End-to-End Correlation Tracking With Enhanced Multi-Level Feature Fusion

Semantic‐aware Spatial Regularization Correlation Filter for Visual Tracking

CorrFormer: Context-aware tracking with cross-correlation and transformer

Learning Bi-Grained Cross-Correlation Siamese Networks for Visual Tracking

Adaptive Weighted CNN Features Integration for Correlation Filter Tracking.

An object tracking framework with recapture based on correlation filters and Siamese networks

A Collaborative Visual Tracking Architecture for Correlation Filter and Convolutional Neural Network Learning

Correlation-Based Transformer Tracking

Combined Correlation Filters With Siamese Region Proposal Network For Visual Tracking

When Correlation Filters Meet Fully-Convolutional Siamese Networks for Distractor-Aware Tracking

Aggregating Correlation Filter with Multiple Kernels Learning for Robust Visual Object Tracking.

When Correlation Filters Meet Siamese Networks for Real-Time Complementary Tracking.

DCFNet++: More Advanced Correlation Filters Network for Real-Time Object Tracking

Correlation Filter Tracker with Siamese: A Robust and Real-Time Object Tracking Framework.

Online Visual Tracking Via Correlation Filter with Convolutional Networks

The Multi-task Fully Convolutional Siamese Network with Correlation Filter Layer for Real-Time Visual Tracking

Correlation-Embedded Transformer Tracking: A Single-Branch Framework