Object Tracking with Mask-Constrained Spatial Context.

Yiyou Guo,Hong Huo,Tao Fang

DOI: https://doi.org/10.1117/1.jei.27.6.069801

IF: 0.829

2018-01-01

Journal of Electronic Imaging

Abstract:Abstract. This publisher’s note corrects a publication error with the paper.

What problem does this paper attempt to address?

Exploit Spatiotemporal Contextual Information for 3D Single Object Tracking Via Memory Networks

Jongwon Ra,MengMeng Wang,Jianbiao Mei,Shanqi Liu,Yu Yang,Yong Liu

DOI: https://doi.org/10.1109/3dv62453.2024.00050

2024-01-01

Abstract:The point cloud-based 3D single object tracking plays an indispensable role in autonomous driving. However, the application of 3D object tracking in the real world is still challenging due to the inherent sparsity and self-occlusion of point cloud data. Therefore, it is necessary to exploit as much useful information from limited data as we can. Since 3D object tracking is a video-level task, the appearance of objects changes gradually over time, and there is rich spatiotemporal contextual information among historical frames. However, existing methods do not fully utilize this information. To address this, we propose a new method called SCTrack, which utilizes a memory-based paradigm to exploit spatiotemporal contextual information. SCTrack incorporates both long-term and short-term memory banks to store the spatiotemporal features of targets from historical frames. By doing so, the tracker can benefit from the entire video sequence and make more informed predictions. Additionally, SCTrack extracts the mask prior to augmenting the target representation, improving the target-background discriminability. Extensive experiments on KITTI, nuScenes, and Waymo Open datasets verify the effectiveness of our proposed method.
Correction to: Spatio Temporal Correlation Graph for Association Enhancement in Multi object Tracking

Zhijie Zhong,Hao Sheng,Yang Zhang,Yubin Wu,Jiahui Chen,Wei Ke

DOI: https://doi.org/10.1007/978-3-030-29551-6_75

2019-01-01

Abstract:Unfortunately the authors of this contribution missed to add an acknowledgment. The acknowledgment should read as follows: Acknowledgement: This study is partially supported by the National Key R&D Program of China (No.2017YFC0806500), the National Natural Science Foundation of China (No.61861166002), the Science and Technology Development Fund of Macau SAR (File no. 0001/2018/AFJ) Joint Scientific Research Project, the Macao Science and Technology Development Fund (No.138/2016/A3), the Fundamental Research Funds for the Central Universities, the Open Fund of the State Key Laboratory of Software Development Environment (No. SKLSDE-2019ZX-04) and the China Scholarship Council State-Sponsored Scholarship Program (Grant No. 201806025026). Thank you for the support from HAWKEYE Group.
Spatio-Temporal clustering model for multi-object tracking through occlusions

Lei Zhang,Qing Wang

DOI: https://doi.org/10.1007/978-3-642-37431-9_14

2012-01-01

Abstract:The occlusion in dynamic or clutter scene is a critical issue in multi-object tracking. Using latent variable to formulate this problem, some methods achieved state-of-the-art performance, while making an exact solution computationally intractable. In this paper, we present a hierarchical association framework to address the problem of occlusion in a complex scene taken by a single camera. At the first stage, reliable tracklets are obtained by frame-to-frame association of detection responses in a flow network. After that, we propose to formulate tracklets association problem in a spatio-temporal clustering model which presents the problem as faithfully as possible. Due to the important role that affinity model plays in our formulation, we then construct a sparsity induced affinity model under the assumption that a detection sample in a tracklet can be efficiently represented by another tracklet belonging to the same object. Furthermore, we give a near-optimal algorithm based on globally greedy strategy to deal with spatio-temporal clustering, which runs linearly with the number of tracklets. We quantitatively evaluate the performance of our method on three challenging data sets and achieve a significant improvement compared to state-of-the-art tracking systems.
Multitarget Occlusion Tracking With 3-D Spatio-Temporal Context Graph Model

Qin Wan,Ruifeng Lv,Yueping Xiao,Zhi Li,Xiaolin Zhu,Yaonan Wang,Haiqiao Liu,Zilong Zeng

DOI: https://doi.org/10.1109/jsen.2023.3303691

IF: 4.3

2023-09-15

IEEE Sensors Journal

Abstract:In complex tracking scenarios of multitarget occlusion, especially when the target is seriously occluded, feature loss can lead to tracking failure. In this article, we develop a multitarget occlusion tracking algorithm based on a 3-D spatio-temporal context graph model, which can fully utilize the targets' RGB-D information. Specifically, we first extract target's 3-D features containing appearance, motion, spatial, and temporal features. Then, the spa'Io-temporal context features between different targets are also be acquired. Furthermore, the attribute relational graph (ARG) model with target's 3-D spatio-temporal context features is constructed, which combines spatial features in the current frame and temporal features between consecutive frames. Moreover, if the target is determined to be occluded according to the matching results in the current frame, the probability relaxation method is used to assign the optimal tracking label to clustering blocks in occlusion area by matching these ARG models, which can address the multitarget cross occlusion problem. Extensive experiments on the Princeton Tracking Benchmark Datasets clearly demonstrate that our method can achieve superior performance against the typical algorithms proposed in recent years.

engineering, electrical & electronic,instruments & instrumentation,physics, applied
Tracking Objects Using Shape Context Matching

Zhao Liu,Hui Shen,Guiyu Feng,Dewen Hu

DOI: https://doi.org/10.1016/j.neucom.2011.11.012

IF: 6

2012-01-01

Neurocomputing

Abstract:In this paper we propose a novel tracking method, which provides accurately segmented object boundaries. The first step of the proposed method is to model the object and background using Gaussian mixture model (GMM), and extract a rough contour according to the object edge features. And then the states of the object, including translation, rotation and scale, are estimated using shape context matching. Finally, we take an elastic shape matching method to extract the exact contour. The proposed method is robust enough for tracking object with translation, rotation, scale change and partial occlusion, and it can also be used for real-time tracking applications. Experiments on both synthetic and real world video sequences demonstrate the effectiveness of the proposed method.
Object tracking using spatio-temporal tracklet association

WANG Jiang-feng,ZHANG Mao-jun,XIONG Zhi-hui,BAO Wei-dong

DOI: https://doi.org/10.3969/j.issn.1001-3695.2011.03.105

2011-01-01

Abstract:Noise segmentation,partial and full object occlusion may result in tracking failure.In order to solve these problems,proposed an object tracking algorithm by associating tracklets with fundamental spatio-temporal constraints.First detect moving objects,then generate tracklet,and grow these tracklets to find the best spatial and temporal association of observations.Experiments prove the proposed method can successfully track multiple moving vehicles and persons under occlusion,noisy detections and split-merge situations.
Robust Visual Tracking Via Context Objects Computing

Zhongqian Sun,Hongxun Yao,Shengping Zhang,Xin Sun

DOI: https://doi.org/10.1109/icip.2011.6116564

2011-01-01

Abstract:Occlusions are challenging issue for robust visual tracking. In this paper, motivated by the fact that a tracked object is usual- ly embedded into context that provides useful information for estimating the target, we propose a novel tracking algorithm named Tracking with Context Prediction (TCP). The context here includes the neighboring objects and specific parts of tar- get. The proposed method simultaneously track the target and context objects using the existing tracking methods. The positions of the context objects are used to predict the position of the target. Thus, the target can be stably tracked even when it is partially or fully occluded. By computing the probability of each prediction being target, our algorithm allows the drifting of context objects during tracking and do not require predictions from all context objects are correct. Experiments on challenging sequences show significant improvements especially in the case of occlusions and appearance changes.
Robust Visual Tracking With Context-Based Active Occlusion Recognition

Yueyang Gu,Yu Qiao,Kuan Xu,Hang Xu,Xingqi Fang

DOI: https://doi.org/10.1109/ICASSP40776.2020.9053630

2020-01-01

Abstract:Occlusion is a great challenge for target model update in visual tracking. The target template may be corrupted by non-object information in the process of online learning due to occlusion. In this paper, we propose a context-based active occlusion recognition framework that can be integrated with various tracking approaches. The basic idea is to recognize the occlusion patches by actively tracking context patches and distinguish them from target based on a target model that integrates the information of both target and context. The framework consists of a context patch tracker, an occlusion patch recognizer and a local target template updater. The context patch tracker locates the context patches. The occlusion patch recognizer identifies the context patches occluding target. The local target template updater updates only non-occluded regions of the target template. In this way, the target template not only learns current target features but also avoids context corruption caused by occlusion. The experimental results show that our framework can improve the tracking performance in cases of heavy occlusion.
WITHDRAWN: Incremental Subspace and Probability Mask Constrained Tracking in Smart and Autonomous Systems

Wang Hongqing,Xu Tingfa,Guo Jie,Rao Zhitao,Shi Guokai

DOI: https://doi.org/10.1016/j.patcog.2017.06.034

IF: 8

2017-01-01

Pattern Recognition

Abstract:This article has been withdrawn at the request of the editor and publisher. The publisher regrets that an error occurred which led to the premature publication of this paper. This error bears no reflection on the article or its authors. The publisher apologizes to the authors and the readers for this unfortunate error. The full Elsevier Policy on Article Withdrawal can be found at http://www.elsevier.com/locate/withdrawalpolicy.
Oversaturated part-based visual tracking via spatio-temporal context learning.

Wei Liu,Jicheng Li,Zhiguang Shi,Xiaotian Chen,Xiao Chen

DOI: https://doi.org/10.1364/AO.55.006960

IF: 1.9

2016-01-01

Applied Optics

Abstract:Partial occlusion is one of the key challenging factors in a robust visual tracking method. To solve this issue, part-based trackers are widely explored; most of them are computationally expensive and therefore infeasible for real-time applications. Context information around the target has been used in tracking, which was recently renewed by a spatio-temporal context (STC) tracker. The fast Fourier transform adopted in STC equips it with high efficiency. However, the global context used in STC alleviates the performance when dealing with occlusion. In this paper, we propose an oversaturated part-based tracker based on spatio-temporal context learning, which tracks objects based on selected parts with spatio-temporal context learning. Furthermore, a structural layout constraint and a novel model update strategy are utilized to enhance the tracker's anti-occlusion ability and to deal with other appearance changes effectively. Extensive experimental results demonstrate our tracker's superior robustness against the original STC and other state-of-art methods. (C) 2016 Optical Society of America
Spatio-Temporal Contextual Learning for Single Object Tracking on Point Clouds

Jiantao Gao,Xu Yan,Weibing Zhao,Zhen Lyu,Yinghong Liao,Chaoda Zheng

DOI: https://doi.org/10.1109/TNNLS.2022.3233562

Abstract:Single object tracking (SOT) is one of the most active research directions in the field of computer vision. Compared with the 2-D image-based SOT which has already been well-studied, SOT on 3-D point clouds is a relatively emerging research field. In this article, a novel approach, namely, the contextual-aware tracker (CAT), is investigated to achieve a superior 3-D SOT through spatially and temporally contextual learning from the LiDAR sequence. More precisely, in contrast to the previous 3-D SOT methods merely exploiting point clouds in the target bounding box as the template, CAT generates templates by adaptively including the surroundings outside the target box to use available ambient cues. This template generation strategy is more effective and rational than the previous area-fixed one, especially when the object has only a small number of points. Moreover, it is deduced that LiDAR point clouds in 3-D scenes are often incomplete and significantly vary from frame to another, which makes the learning process more difficult. To this end, a novel cross-frame aggregation (CFA) module is proposed to enhance the feature representation of the template by aggregating the features from a historical reference frame. Leveraging such schemes enables CAT to achieve a robust performance, even in the case of extremely sparse point clouds. The experiments confirm that the proposed CAT outperforms the state-of-the-art methods on both the KITTI and NuScenes benchmarks, achieving 3.9% and 5.6% improvements in terms of precision.
Online Multiple Object Tracking Via Exchanging Object Context.

Hongyang Yu,Lei Qin,Qingming Huang,Hongxun Yao

DOI: https://doi.org/10.1016/j.neucom.2018.02.068

IF: 6

2018-01-01

Neurocomputing

Abstract:Multiple object tracking is a key problem for many computer vision applications such as video surveillance, advanced driver assistance or animation. Most of existing tracking-by-detection methods are mainly based on object appearances and motions. However, the contextual information around the target has not been fully exploited. In this paper, we pay more attention to the contextual information and propose an Exchanging Object Context (EOC) model, which takes full advantage of the context information. Specifically, we implement an efficient and accurate online multiple object tracking algorithm with a novel affinity measure to associate detections. This measure calculates the similarity between targets and detections with the background smoothness after exchanging the contexts between detections and targets, using a novel color histogram descriptor. We refine the bounding boxes by measuring the context changes. Extensive experimental results on two public benchmarks demonstrate the effectiveness of the proposed tracking method with comparisons to several state-of-the-art trackers.
Visual Tracking Via Part-Based Model with Auxiliary Objects

Runpeng Cui,Guangdong Hou,Changshui Zhang

DOI: https://doi.org/10.1109/smc.2015.523

2015-01-01

Abstract:In most model-free tracking algorithms, context of the target is usually taken as source of negative examples for training the appearance model and thus not fully utilized. In fact, the target is embedded in its context, and there are spatial constraints between them with the potential motion correlations and relative locations. This paper presents a part-based model to describe the spatial constraints between the target object and its context. Auxiliary objects are selected as parts of the part based model to represent the context information. These auxiliary objects should have some properties: (i) co-occurrence with the target, (ii) having distinct appearance, and (iii) having homogeneous motion as an invisible visual region. A template-based model concerning about the object appearance is implemented to help estimate the reliability of tracking and make the part-based model updated properly. We test the proposed method on a recent benchmark with 50 video sequences evaluated, and a superior performance is achieved compared with state-of-the-arts.
VisuaI Tracking Using Adaptive Structure ModeI Based on Spatio-TemporaI Context

Zheng-yu ZHU,Shuai LI,Wei ZHU

DOI: https://doi.org/10.3969/j.issn.1007-1423.2016.20.001

2016-01-01

Abstract:Object tracking is one of the basic problems in the field of computer vision. There are many algorithms presented, and STC is a quite novel one. But the STC tracking method canˊt deal with model drift problem. To overcome this weakness, proposes an algorithm using adaptive structure model based on STC. This algorithm takes a set number of snapshots of normal template as snapshot templates, and saves them to snapshot set. When one of the snapshot templates gets an enough better outcome than normal template, the algorithm uses the snapshot template to roll back the normal template, which can effectively enhance the tracking robustness and keep accurate even when object suffers all kinds of interferences such as fast motion, in-plane rotation, motion blur, severe occlusion and so on. The experi-ment on Tracker Benchmark v1.0 dataset compared to STC shows that the average accuracy is improved from 38.61%to 42.02%, the av-erage center location error is decreased from 85.57 to 62.78 and the average frame rate decreased from 45.89 fps to 36.64 fps. It still meets the requirements of real-time tracking. In conclusion, this algorithm can accomplish the real-time tracking steadily even when dis-turbed by all kinds of interferences.
Object Tracking Using Spatio-Temporal Future Prediction

Yuan Liu,Ruoteng Li,Robby T. Tan,Yu Cheng,Xiubao Sui

DOI: https://doi.org/10.48550/arXiv.2010.07605

2020-10-15

Computer Vision and Pattern Recognition

Abstract:Occlusion is a long-standing problem that causes many modern tracking methods to be erroneous. In this paper, we address the occlusion problem by exploiting the current and future possible locations of the target object from its past trajectory. To achieve this, we introduce a learning-based tracking method that takes into account background motion modeling and trajectory prediction. Our trajectory prediction module predicts the target object's locations in the current and future frames based on the object's past trajectory. Since, in the input video, the target object's trajectory is not only affected by the object motion but also the camera motion, our background motion module estimates the camera motion. So that the object's trajectory can be made independent from it. To dynamically switch between the appearance-based tracker and the trajectory prediction, we employ a network that can assess how good a tracking prediction is, and we use the assessment scores to choose between the appearance-based tracker's prediction and the trajectory-based prediction. Comprehensive evaluations show that the proposed method sets a new state-of-the-art performance on commonly used tracking benchmarks.
Mask Guided Spatial-Temporal Fusion Network for Multiple Object Tracking.

Shuangye Zhao,Yubin Wu,Shuai Wang,Wei Ke,Hao Sheng

DOI: https://doi.org/10.1109/ICIP46576.2022.9898054

2022-01-01

ICIP

Abstract:Multi-object trackers make the association almost perfectly when no occlusion occurred between two or more targets. However, it is hard to extract reliable features on account of partial occlusion caused by a nearby object, which often leads to tracking failure. In this paper, we utilize mask to guide attention of the neural network in order to focus on the visible part of the target and design a tracklet-level feature extraction method. Then, a tracking framework is proposed based on a mask guided fusion network and multi-hypothesis tracking algorithm. Comprehensive evaluation on the MOT17 dataset shows that our approach achieves competitive results.
An Improved Target Tracking Algorithm Based on Spatio-Temporal Context under Occlusions

Xin Yang,Songyan Zhu,Dake Zhou,Yifan Zhang

DOI: https://doi.org/10.1007/s11045-019-00664-5

IF: 2.03

2019-01-01

Multidimensional Systems and Signal Processing

Abstract:Target tracking is a popular but challenging problem in computer vision field. Due to many aggravating factors such as position transformation, illumination, occlusion, it is difficult to achieve robust target tracking. According to the above constraints, an improved target tracking algorithm based on spatio-temporal context (STC) under occlusions is proposed. On the basis of STC, the proposed method introduces a novel mechanism for dealing with occlusion, the scale update, and the learning rate update to reduce the error update of the model and restrain error accumulation. As a consequence, the tracking performance can be improved efficiently. Extensive experimental results show that our algorithm outperforms the original STC algorithm and some other state-of-the-art algorithms.
Incremental Subspace and Probability Mask Constrained Tracking in Smart and Autonomous Systems (withdrawn Publication. See Vol. 76, Pg. 764, 2018) (withdrawn Publication. See Vol. 76, Pg. 764, 2018)

Hongqing Wang,Tingfa Xu,Jie Guo,Zhitao Rao,Guokai Shi

DOI: https://doi.org/10.1016/j.patcog.2017.06.034

IF: 8

2017-01-01

Pattern Recognition

Abstract:In this paper, we propose a novel incremental subspace and probability mask constrained sparse representation model in Smart and Autonomous Systems (SAS). In contrast to traditional sparse representation based tracking methods, the proposed model uses the trivial templates to model both reconstruction errors caused by sparse representation and the Eigen subspace representation. Besides, to alleviate the fact that the trivial templates can be activated to represent any image patch, we further propose to constrain the trivial templates with a probability mask. A unified objective function is proposed and a customized APG method is developed to effectively solve the optimization problem. In addition, a robust observation likelihood metric is proposed and a Bayes rule based color histogram model is proposed to construct the probability mask. Numerous qualitative and quantitative evaluations demonstrate that our tracker outperforms the state-of-the-art trackers in a wide range of tracking scenarios. (C) 2017 Elsevier Ltd. All rights reserved.
Visual Object Tracking With Mutual Affinity Aligned to Human Intuition

Guotian Zeng,Bi Zeng,Qingmao Wei,Huiting Hu,Hong Zhang

DOI: https://doi.org/10.1109/tmm.2024.3405654

IF: 7.3

2024-10-19

IEEE Transactions on Multimedia

Abstract:Single-object tracking generally advances by incrementally determining the tracked target's position through interactions between the search region and the template. However, the template provides less information than does the search region in terms of both temporal cues and spatial resolution. To alleviate this imbalance, we introduce an anthropic tracking framework, MATrack (Mutual Affinity Tracker), which explicitly strengthens weak template information and implicitly reduces background clutter through interactions between multiple templates and the search region. Additionally, we propose a coarse-to-fine localization approach that combines the benefits of corner-based and center-based methods. This approach enables us to simultaneously update the most recent state and background information without two-stage training. MATrack achieves state-of-the-art performance on multiple test benchmarks, including GOT-10k, LASOT, TrackingNet, OTB-100, UAV123, and NFS30. Among these benchmarks, MATrack-320's performance stands out, particularly in the short-term tracking dataset GOT-10k, where it achieves an accuracy overlap (AO) of 77.3. We also conduct comprehensive quantitative and qualitative evaluations to demonstrate that our method significantly outperforms other state-of-the-art approaches.

computer science, information systems,telecommunications, software engineering
An Efficient And Robust Visual Tracking Via Color-Based Context Prior Model

Chunlei Yu,Baojun Zhao,Zengshuo Zhang,Maowen Li

DOI: https://doi.org/10.1007/978-3-319-71607-7_7

2017-01-01

Abstract:Real-time object tracking has been widely applied to time-critical multimedia fields such as surveillance and human-computer interaction. It is a challenge to balance accuracy and speed in tracking. Spatio-Temporal Context tracker (STC) formulates the spatio-temporal relationship between the object and its surrounding background, and achieves good performance in accuracy and speed. However, the context prior model only utilizes the grayscale feature which is not efficient. When the target is not obvious in the context, or the context exists a similar interference compared to the target, STC tracker drifts from the target. To solve the problem, we exploit the standard color histograms of the context to build a discriminative context prior model. More specifically, we utilize an effective lookup-table to compute the prior context model at a low computational cost. Finally, extensive experiments on challenging sequences show the effectiveness and robustness of our proposed method.

Object Tracking with Mask-Constrained Spatial Context.

Exploit Spatiotemporal Contextual Information for 3D Single Object Tracking Via Memory Networks

Correction to: Spatio Temporal Correlation Graph for Association Enhancement in Multi object Tracking

Spatio-Temporal clustering model for multi-object tracking through occlusions

Multitarget Occlusion Tracking With 3-D Spatio-Temporal Context Graph Model

Tracking Objects Using Shape Context Matching

Object tracking using spatio-temporal tracklet association

Robust Visual Tracking Via Context Objects Computing

Robust Visual Tracking With Context-Based Active Occlusion Recognition

WITHDRAWN: Incremental Subspace and Probability Mask Constrained Tracking in Smart and Autonomous Systems

Oversaturated part-based visual tracking via spatio-temporal context learning.

Spatio-Temporal Contextual Learning for Single Object Tracking on Point Clouds

Online Multiple Object Tracking Via Exchanging Object Context.

Visual Tracking Via Part-Based Model with Auxiliary Objects

VisuaI Tracking Using Adaptive Structure ModeI Based on Spatio-TemporaI Context

Object Tracking Using Spatio-Temporal Future Prediction

Mask Guided Spatial-Temporal Fusion Network for Multiple Object Tracking.

An Improved Target Tracking Algorithm Based on Spatio-Temporal Context under Occlusions

Incremental Subspace and Probability Mask Constrained Tracking in Smart and Autonomous Systems (withdrawn Publication. See Vol. 76, Pg. 764, 2018) (withdrawn Publication. See Vol. 76, Pg. 764, 2018)

Visual Object Tracking With Mutual Affinity Aligned to Human Intuition

An Efficient And Robust Visual Tracking Via Color-Based Context Prior Model