A Survey on Visual Tracking Via Convolutional Neural Networks

Jia Zhang,Lei Yang,Xiaoyu Wu

DOI: https://doi.org/10.1109/compcomm.2016.7924746

2016-01-01

Abstract:The paper summarized 13 classic methods of visual tracking and 9 methods combined with convolutional neural networks including some latest trackers and compared all these trackers in the same benchmark. After analyzing the results, we had some new idea for existing trackers.

What problem does this paper attempt to address?

Deep visual tracking: Review and experimental comparison

Peixia Li,Dong Wang,Lijun Wang,Huchuan Lu

DOI: https://doi.org/10.1016/j.patcog.2017.11.007

IF: 8

2018-04-01

Pattern Recognition

Abstract:Recently, deep learning has achieved great success in visual tracking. The goal of this paper is to review the state-of-the-art tracking methods based on deep learning. First, we introduce the background of deep visual tracking, including the fundamental concepts of visual tracking and related deep learning algorithms. Second, we categorize the existing deep-learning-based trackers into three classes according to network structure, network function and network training. For each categorize, we explain its analysis of the network perspective and analyze papers in different categories. Then, we conduct extensive experiments to compare the representative methods on the popular OTB-100, TC-128 and VOT2015 benchmarks. Based on our observations, we conclude that: (1) The usage of the convolutional neural network (CNN) model could significantly improve the tracking performance. (2) The trackers using the convolutional neural network (CNN) model to distinguish the tracked object from its surrounding background could get more accurate results, while using the CNN model for template matching is usually faster. (3) The trackers with deep features perform much better than those with low-level hand-crafted features. (4) Deep features from different convolutional layers have different characteristics and the effective combination of them usually results in a more robust tracker. (5) The deep visual trackers using end-to-end networks usually perform better than the trackers merely using feature extraction networks. (6) For visual tracking, the most suitable network training method is to per-train networks with video information and online fine-tune them with subsequent observations. Finally, we summarize our manuscript and highlight our insights, and point out the further trends for deep visual tracking.

computer science, artificial intelligence,engineering, electrical & electronic
Study on Deep Learning and Its Application in Visual Tracking.

Dan Hu,Xiaohao Yu,Xingshe Zhou,Zhiqiang Hou

DOI: https://doi.org/10.1109/bwcca.2015.63

2015-01-01

Abstract:Inspired by recent advances in deep learning, this paper reviews the deep learning methodologies and its applications in object tracking. To overcome the complexity and low-efficiency of existing full-connected deep learning based tracker, we use a novel convolutional deep belief network (CDBN) with convolution, weights sharing and pooling to have much fewer parameters, in addition to gain translation invariance which would benefit the tracker performance. Empirical evaluation demonstrates our CDBN based tracker outperforms several state-of-the-art methods on an open tracker benchmark.
Online Tracking With Convolutional Neural Networks

Xiaodong Liu,Yue Zhou

DOI: https://doi.org/10.1007/978-3-319-70090-8_22

2017-01-01

Abstract:Convolutional neural networks (CNNs) have recently been widely applied to visual applications, but there are still not much attempts to employ CNNs for object tracking. In this paper, we propose a novel visual tracking method which utilizes the powerful representations of CNNs. We regard the visual tracking as a traditional binary classification task along with an online model update. The binary classification network is pre-trained on ImageNet dataset and fine-tuned on visual tracking benchmark dataset by sequentially training to avoid overfitting. In the tracking process, we conduct a short-term and long-term model update mechanism for adaptiveness and robustness, respectively. Extensive experiments on two visual tracking datasets demonstrate that our algorithm is comparable to state-of-art methods in terms of accuracy and robustness.
Visual Tracking with Fully Convolutional Networks

Lijun Wang,Wanli Ouyang,Xiaogang Wang,Huchuan Lu

DOI: https://doi.org/10.1109/iccv.2015.357

2015-01-01

Abstract:We propose a new approach for general object tracking with fully convolutional neural network. Instead of treating convolutional neural network (CNN) as a black-box feature extractor, we conduct in-depth study on the properties of CNN features offline pre-trained on massive image data and classification task on ImageNet. The discoveries motivate the design of our tracking system. It is found that convolutional layers in different levels characterize the target from different perspectives. A top layer encodes more semantic features and serves as a category detector, while a lower layer carries more discriminative information and can better separate the target from distracters with similar appearance. Both layers are jointly used with a switch mechanism during tracking. It is also found that for a tracking target, only a subset of neurons are relevant. A feature map selection method is developed to remove noisy and irrelevant feature maps, which can reduce computation redundancy and improve tracking accuracy. Extensive evaluation on the widely used tracking benchmark [36] shows that the proposed tacker outperforms the state-of-the-art significantly.
Visual object tracking: A survey

Fei Chen,Xiaodong Wang,Yunxiang Zhao,Shaohe Lv,Xin Niu

DOI: https://doi.org/10.1016/j.cviu.2022.103508

IF: 4.886

2022-09-01

Computer Vision and Image Understanding

Abstract:Visual object tracking is an important area in computer vision, and many tracking algorithms have been proposed with promising results. Existing object tracking approaches can be categorized into generative trackers, discriminative trackers, and collaborative trackers. Recently, object tracking algorithms based on deep neural networks have emerged and obtained great attention from researchers due to their outstanding tracking performance. To summarize the development of object tracking, a few surveys give analyses on either deep or non-deep trackers. In this paper, we provide a comprehensive overview of state-of-the-art tracking frameworks including both deep and non-deep trackers. We present both quantitative and qualitative tracking results of various trackers on five benchmark datasets and conduct a comparative analysis of their results. We further discuss challenging circumstances such as occlusion, illumination, deformation, and motion blur. Finally, we list the challenges and the future work in this fast-growing field.

computer science, artificial intelligence,engineering, electrical & electronic
A Survey of Visual Tracking

HOU Zhi-Qiang,HAN Chong-Zhao

DOI: https://doi.org/10.16383/j.aas.2006.04.016

2006-01-01

ACTA AUTOMATICA SINICA

Abstract:This paper introduces in detail the research of visual tracking which is a hot spot currently in the domain of computer vision. Firstly, the applications of visual tracking in three areas including visual surveillance, image compression, and 3-D reconstruction are discussed. Secondly, the state of the art about visual tracking is introduced, especially the main approaches of visual tracking are shown. In order to explain these methods clearly, the problems of visual tracking are classified. Then two ways to research the visual tracking problem are presented. And the visual tracking algorithms are classified into four classes: the area-based methods, feature-based methods, deformable-template-based methods and model-based methods. Finally, from the point of view of control theory, the difficulties of visual tracking are discussed that the algorithms should have robustness accuracy and be fast. Meanwhile, some future directions of visual tracking are also addressed shortly.
CFNN: Correlation Filter Neural Network for Visual Object Tracking

Yang Li,Zhan Xu,Jianke Zhu

DOI: https://doi.org/10.24963/ijcai.2017/309

2017-01-01

Abstract:Albeit convolutional neural network (CNN) has shown promising capacity in many computer vision tasks, applying it to visual tracking is yet far from solved. Existing methods either employ a large external dataset to undertake exhaustive pre-training or suffer from less satisfactory results in terms of accuracy and robustness. To track single target in a wide range of videos, we present a novel Correlation Filter Neural Network architecture, as well as a complete visual tracking pipeline, The proposed approach is a special case of CNN, whose initialization does not need any pre-training on the external dataset. The initialization of network enjoys the merits of cyclic sampling to achieve the appealing discriminative capability, while the network updating scheme adopts advantages from back-propagation in order to capture new appearance variations. The tracking pipeline integrates both aspects well by making them complementary to each other. We validate our tracker on OTB-2013 benchmark. The proposed tracker obtains the promising results compared to most of existing representative trackers.
Visual Tracking Based on Convolutional Neural Network

Xiuzhi Li,Kai Jiang,Songmin Jia,Xiangyin Zhang,Yanjun Sun

DOI: https://doi.org/10.1109/cac.2018.8623581

2018-01-01

Abstract:The tracking object is one of the important research directions in the field of computer vision and plays an important role in intelligent video monitoring. In this paper, a visual tracking method based on deep learning object detection is proposed. There are many functions required in human object tracking tasks, including the drive of hight-definition panoramic camera, real-time video streaming protocol of RTSP, object detection based on deep convolution neural network, ROI selection of interest area, dynamic object tracking of KF, and online video distribution of human coordinates through SOCKET communication.
Visual Tracking Based on Convolutional Deep Belief Network

Dan Hu,Xingshe Zhou,Junjie Wu

DOI: https://doi.org/10.1007/978-3-319-23216-4_8

2015-01-01

Abstract:Visual tracking is an important task within the field of computer vision. Recently, deep neural networks have gained significant attention thanks to their success on learning image features. But the existing deep neural networks applied in visual tracking are full-connected complicated architectures with large amount of redundant parameters that would be low efficiently to learn. We tackle this problem by using a novel convolutional deep belief network (CDBN) with convolution, weights sharing and pooling to have much fewer parameters to learn, in addition to gain translation invariance which would benefit the tracker performance. Theoretical analysis and experimental evaluations on an open tracker benchmark demonstrate our CDBN based tracker is more accurate by improving tracking success rate 22.6% and tracking precision 62.8% on average, while maintaining low computation cost by reduces the number of parameters to 44.4%, compared to DLT, another well-known deep learning tracker. Meanwhile, our tracker can achieve real-time performance by a graphics processing unit (GPU) speedup of 2.61 times on average and up to 3.08 times.
Detection based visual tracking with convolutional neural network.

Yong Wang,Xinbin Luo,Lu Ding,Shan Fu,Xian Wei

DOI: https://doi.org/10.1016/j.knosys.2019.03.012

IF: 8.139

2019-01-01

Knowledge-Based Systems

Abstract:In this paper, we propose a detection strategy based visual object tracking algorithm. We employ multiple trackers using layers of deep convolutional neural network (CNN) features. Each tracker which is correlation filter based tracking framework tracks an object forwardly and then backwardly. By analyzing the forward and backward trajectories, we measure the robustness of tracking results. A detection strategy which is based on locally adaptive regression kernels (LARK) feature is carried out according to the robustness of tracking result. Target can be located from the provided candidates. Extensive experimental results show that the proposed method improves the accuracy and robustness of tracking, achieving state-of-the-art results on several recent benchmark datasets.
Deep Learning Based Visual Tracking: A Review

Chun-bao Li,Bo Yang,Chun-hu Li

DOI: https://doi.org/10.12783/dtcse/smce2017/12427

2017-01-01

DEStech Transactions on Computer Science and Engineering

Abstract:Visual tracking, a traditional computer vision task, has been a popular research field in recent decades. As a powerful features learning method, deep learning provides a new way for the realization of visual tracking with higher accuracy and performance. Many novel trackers that based on different network models, including auto-encoder (SAE), convolutional neural network (CNN), recurrent neural networks (RNN) deep reinforcement learning (DRL) and the fusion of them were proposed by researchers in the literatures. This paper presents a comprehensive survey on deep learning based visual tracking algorithms
Multiple Convolutional Neural Networks for Online Visual Tracking.

Tianli Sun,Cairong Zhao,Kang Chen

DOI: https://doi.org/10.1109/ccis.2018.8691134

2018-01-01

Abstract:The paper presents a new online visual tracking method based on convolutional neural network (CNN). Our algorithm consists of multiple CNN tracker units focusing on different object features, and each unit is trained with a proper number of domains determined by our method. For the model update process, traditional CNN online re-training is avoided, replaced by a timely tracker switching strategy. At each moment, only one main tracker predicts and others stand by to ensure efficiency. The experimental results show that our method can track objects in most scenarios, and implicate that our CNN model has a generalization ability in distinguishing target and background.
Continuity-Discrimination Convolutional Neural Network for Visual Object Tracking

Shen Li,Bingpeng Ma,Hong Chang,Shiguang Shan,Xilin Chen

DOI: https://doi.org/10.48550/arXiv.2104.08739

2021-04-18

Abstract:This paper proposes a novel model, named Continuity-Discrimination Convolutional Neural Network (CD-CNN), for visual object tracking. Existing state-of-the-art tracking methods do not deal with temporal relationship in video sequences, which leads to imperfect feature representations. To address this problem, CD-CNN models temporal appearance continuity based on the idea of temporal slowness. Mathematically, we prove that, by introducing temporal appearance continuity into tracking, the upper bound of target appearance representation error can be sufficiently small with high probability. Further, in order to alleviate inaccurate target localization and drifting, we propose a novel notion, object-centroid, to characterize not only objectness but also the relative position of the target within a given patch. Both temporal appearance continuity and object-centroid are jointly learned during offline training and then transferred for online tracking. We evaluate our tracker through extensive experiments on two challenging benchmarks and show its competitive tracking performance compared with state-of-the-art trackers.

Computer Vision and Pattern Recognition
Visual Tracking Based on Semantic and Similarity Learning

Yufei Zha,Min Wu,Zhuling Qiu,Wangsheng Yu

DOI: https://doi.org/10.1049/iet-cvi.2018.5826

IF: 1.484

2019-01-01

IET Computer Vision

Abstract:We present a method by combining the similarity and semantic features of a target to improve tracking performance in video sequences. Trackers based on Siamese networks have achieved success in recent competitions and databases through learning similarity according to binary labels. Unfortunately, such weak labels result in limiting the discriminative ability of the learned feature, thus it is difficult to identify the target itself from the distractors that have the same class. The authors observe that the inter‐class semantic features benefit to increase the separation between the target and the background, even distractors. Therefore, they proposed a network architecture which uses both similarity and semantic branches to obtain more discriminative features for locating the target accuracy in new frames. The large‐scale ImageNet VID dataset is employed to train the network. Even in the presence of background clutter, visual distortion, and distractors, the proposed method still maintains following the target. They test their method with the open benchmarks OTB and UAV123. The results show that their combined approach significantly improves the tracking ability relative to trackers using similarity or semantic features alone.
Online Visual Tracking Via Correlation Filter with Convolutional Networks

Zheng Li,Jianfei Yang,Juan Zha,Chang-Dong Wang,Weishi Zheng

DOI: https://doi.org/10.1109/vcip.2016.7805476

2016-01-01

Abstract:Robust online visual tracking is a challenging task of the computer vision due to its violent variation within the video sequences. To approach these issues, deep networks have been applied in order to improve accuracy and correlation filter based trackers perform excellent efficiency and adaptation to scale. In this paper, we present a novel method with convolutional networks and correlation filter. A simple two-layer convolutional network is constructed to learn robust representations, which encode the inner geometric layout and local structural information, and the tracking framework resorts to learning discriminative correlation filters based on them. For our method satisfies both veracity and efficiency by finding a compromise between two theories, it performs favorably against several state-of-the-art methods with 50 public challenging videos.
Robust Visual Tracking Method via Deep Learning

Jun-Yu GAO,Xiao-Shan YANG,Tian-Zhu ZHANG,Chang-Sheng XU

DOI: https://doi.org/10.11897/SP.J.1016.2016.01419

2016-01-01

Abstract:The traditional tracking methods (e.g.L1 tracker)generally adopt the pixel values as feature representation,and ignore the deep visual features of image patches.In a fixed video scene of the real world,we realize that we can usually find an area where the targets have clear appearance and are easy to distinguish.Therefore,in this paper,we select a region in each video to construct training set for deep model learning.In the proposed deep model,we design a deep convolutional neural network which has two symmetrical paths with the shared weights.The goal of the proposed deep network is to reduce the difference between the features of a target out of the region and in the region.As a result,the learned deep network can enhance the appearance feature of targets and benefit the trackers that utilize low-level features,such as L1 tracker.Finally, we utilize this pre-trained deep convolutional network in the L1 tracker to extract features for sparse representation.Consequently,our method achieves the robustness in tracking for handling the challenges such as occlusion and illumination changes.We evaluate the proposed approach on 25 challenging videos against with 9 state-of-the-art trackers.The extensive results show that the proposed algorithm is 0.11 higher than the second best with average overlap,and is 1.0 lower than the second best with the average center location errors.
Once for All: a Two-flow Convolutional Neural Network for Visual Tracking

Kai Chen,Wenbing Tao

DOI: https://doi.org/10.1109/tcsvt.2017.2757061

IF: 5.859

2018-01-01

IEEE Transactions on Circuits and Systems for Video Technology

Abstract:The main challenges of visual object tracking arise from the arbitrary appearance of the objects that need to be tracked. Most existing algorithms try to solve this problem by training a new model to regenerate or classify each tracked object. As a result, the model needs to be initialized and retrained for each new object. In this paper, we propose to track different objects in an object-independent approach with a novel two-flow convolutional neural network (YCNN). The YCNN takes two inputs (one is an object image patch, the other is a larger searching image patch), then outputs a response map which predicts how likely and where the object would appear in the search patch. Unlike the object-specific approaches, the YCNN is actually trained to measure the similarity between the two image patches. Thus, this model will not be limited to any specific object. Furthermore, the network is end-to-end trained to extract both shallow and deep dedicated convolutional features for visual tracking. And once properly trained, the YCNN can be used to track all kinds of objects without further training and updating. As a result, our algorithm is able to run at a very high speed of 45 frames-per-second. The effectiveness of the proposed algorithm can also be proved by the experiments on two popular data sets: OTB-100 and VOT-2014.
Survey of Visual Tracking Algorithms

杨戈,刘宏

DOI: https://doi.org/10.3969/j.issn.1673-4785.2010.02.001

2010-01-01

Abstract:Progress in information technology and intelligent systems has brought computer vision to the forefront of IT research.Visual tracking is currently one of the most active research topics in computer vision.The state of research in visual tracking algorithms was examined for this paper.The authors reviewed the common visual tracking algorithms and their main mathematical methods.The examination included region-based tracking algorithms,active contour-based tracking algorithms,feature-based tracking algorithms,model-based tracking algorithms,parameter estimate algorithms and non-parameter density estimate algorithms.Significant issues needing further study were then discussed.
Robust Visual Track Using an Ensemble Cascade of Convolutional Neural Networks

Dan Hu,Xingshe Zhou,Xiaohao Yu,Zhiqiang Hou

DOI: https://doi.org/10.1117/12.2228001

2015-01-01

Abstract:Convolutional Neural Networks (CNN) have dramatically boosted the performance of various computer vision tasks except visual tracking due to the lack of training data. In this paper, we pre-train a deep CNN offline to classify the 1 million images from 256 classes with very leaky non-saturating neurons for training acceleration, which is transformed to a discriminative classifier by adding an additional classification layer. In addition, we propose a novel approach for combining increasingly our CNN classifiers in a "cascade" structure through a modification of the AdaBoost framework, and then transfer the selected discriminative features from the ensemble of CNN classifiers to the robust visual tracking task, by updating online to robustly discard the background regions from promising object-like region to cope with appearance changes of the target. Extensive experimental evaluations on an open tracker benchmark demonstrate outstanding performance of our tracker by improving tracking success rate and tracking precision on an average of 9.2% and 13.9% at least over other state-of-the-art trackers.
High Performance Visual Object Tracking with Unified Convolutional Networks

Zhu Zheng,Zou Wei,Huang Guan,Du Dalong,Huang Chang

2019-01-01

Abstract: Convolutional neural networks (CNN) based tracking approaches have shown favorable performance in recent benchmarks. Nonetheless, the chosen CNN features are always pre-trained in different tasks and individual components in tracking systems are learned separately, thus the achieved tracking performance may be suboptimal. Besides, most of these trackers are not designed towards real-time applications because of their time-consuming feature extraction and complex optimization details. In this paper, we propose an end-to-end framework to learn the convolutional features and perform the tracking process simultaneously, namely, a unified convolutional tracker (UCT). Specifically, the UCT treats feature extractor and tracking process both as convolution operation and trains them jointly, which enables learned CNN features are tightly coupled with tracking process. During online tracking, an efficient model updating method is proposed by introducing peak-versus-noise ratio (PNR) criterion, and scale changes are handled efficiently by incorporating a scale branch into network. Experiments are performed on four challenging tracking datasets: OTB2013, OTB2015, VOT2015 and VOT2016. Our method achieves leading performance on these benchmarks while maintaining beyond real-time speed.

A Survey on Visual Tracking Via Convolutional Neural Networks

Deep visual tracking: Review and experimental comparison

Study on Deep Learning and Its Application in Visual Tracking.

Online Tracking With Convolutional Neural Networks

Visual Tracking with Fully Convolutional Networks

Visual object tracking: A survey

A Survey of Visual Tracking

CFNN: Correlation Filter Neural Network for Visual Object Tracking

Visual Tracking Based on Convolutional Neural Network

Visual Tracking Based on Convolutional Deep Belief Network

Detection based visual tracking with convolutional neural network.

Deep Learning Based Visual Tracking: A Review

Multiple Convolutional Neural Networks for Online Visual Tracking.

Continuity-Discrimination Convolutional Neural Network for Visual Object Tracking

Visual Tracking Based on Semantic and Similarity Learning

Online Visual Tracking Via Correlation Filter with Convolutional Networks

Robust Visual Tracking Method via Deep Learning

Once for All: a Two-flow Convolutional Neural Network for Visual Tracking

Survey of Visual Tracking Algorithms

Robust Visual Track Using an Ensemble Cascade of Convolutional Neural Networks

High Performance Visual Object Tracking with Unified Convolutional Networks