Abstract:Recently, deep learning has achieved great success in visual tracking. The goal of this paper is to review the state-of-the-art tracking methods based on deep learning. First, we introduce the background of deep visual tracking, including the fundamental concepts of visual tracking and related deep learning algorithms. Second, we categorize the existing deep-learning-based trackers into three classes according to network structure, network function and network training. For each categorize, we explain its analysis of the network perspective and analyze papers in different categories. Then, we conduct extensive experiments to compare the representative methods on the popular OTB-100, TC-128 and VOT2015 benchmarks. Based on our observations, we conclude that: (1) The usage of the convolutional neural network (CNN) model could significantly improve the tracking performance. (2) The trackers using the convolutional neural network (CNN) model to distinguish the tracked object from its surrounding background could get more accurate results, while using the CNN model for template matching is usually faster. (3) The trackers with deep features perform much better than those with low-level hand-crafted features. (4) Deep features from different convolutional layers have different characteristics and the effective combination of them usually results in a more robust tracker. (5) The deep visual trackers using end-to-end networks usually perform better than the trackers merely using feature extraction networks. (6) For visual tracking, the most suitable network training method is to per-train networks with video information and online fine-tune them with subsequent observations. Finally, we summarize our manuscript and highlight our insights, and point out the further trends for deep visual tracking.

An Ensemble of Complementary Models for Deep Tracking

Continuity-Discrimination Convolutional Neural Network for Visual Object Tracking

Robust Visual Tracking Via Multiple Discriminative Models with Object Proposals

Ensemble Tracking Based on CNN

A Complementary Tracking Model with Multiple Features.

Online Object Tracking Based on CNN with Spatial-Temporal Saliency Guided Sampling

Deep Feature Tracking Based on Interactive Multiple Model.

CMAT: Integrating Convolution Mixer and Self-Attention for Visual Tracking

Robust Visual Track Using an Ensemble Cascade of Convolutional Neural Networks

Real-Time Online Tracking Via a Convolution-Based Complementary Model

CVTrack: Combined Convolutional Neural Network and Vision Transformer Fusion Model for Visual Tracking

Complementary Learners for Visual Tracking

Learning Complementary Siamese Networks for Real-Time High-Performance Visual Tracking.

Adaptive feature fusion for visual object tracking

High Performance Visual Object Tracking with Unified Convolutional Networks

Deep Ensemble Tracking.

Robust Visual Tracking with Deep Feature Fusion

Interactive Convolutional Learning for Visual Tracking.

Adaptive Object Tracking with Complementary Models

Deep visual tracking: Review and experimental comparison

One-shot Multi-Object Tracking Using CNN-based Networks with Spatial-Channel Attention Mechanism