[Significance of cardiovascular research within the scope of the total development of medical sciences in East Germany].

H. Klinkmann

1985-07-01

Abstract:

What problem does this paper attempt to address?

Transformer Network for Multi-Person Tracking and Re-Identification in Unconstrained Environment

Hamza Mukhtar,Muhammad Usman Ghani Khan

2023-12-19

Abstract:Multi-object tracking (MOT) has profound applications in a variety of fields, including surveillance, sports analytics, self-driving, and cooperative robotics. Despite considerable advancements, existing MOT methodologies tend to falter when faced with non-uniform movements, occlusions, and appearance-reappearance scenarios of the objects. Recognizing this inadequacy, we put forward an integrated MOT method that not only marries object detection and identity linkage within a singular, end-to-end trainable framework but also equips the model with the ability to maintain object identity links over long periods of time. Our proposed model, named STMMOT, is built around four key modules: 1) candidate proposal generation, which generates object proposals via a vision-transformer encoder-decoder architecture that detects the object from each frame in the video; 2) scale variant pyramid, a progressive pyramid structure to learn the self-scale and cross-scale similarities in multi-scale feature maps; 3) spatio-temporal memory encoder, extracting the essential information from the memory associated with each object under tracking; and 4) spatio-temporal memory decoder, simultaneously resolving the tasks of object detection and identity association for MOT. Our system leverages a robust spatio-temporal memory module that retains extensive historical observations and effectively encodes them using an attention-based aggregator. The uniqueness of STMMOT lies in representing objects as dynamic query embeddings that are updated continuously, which enables the prediction of object states with attention mechanisms and eradicates the need for post-processing.

Computer Vision and Pattern Recognition,Artificial Intelligence,Machine Learning
Exploit the Connectivity: Multi-Object Tracking with TrackletNet

Gaoang Wang,Yizhou Wang,Haotian Zhang,Renshu Gu,Jenq-Neng Hwang

DOI: https://doi.org/10.1145/3343031.3350853

2019-01-01

Abstract:Multi-object tracking (MOT) is an important topic and critical task related to both static and moving camera applications, such as traffic flow analysis, autonomous driving and robotic vision. However, due to unreliable detection, occlusion and fast camera motion, tracked targets can be easily lost, which makes MOT very challenging. Most recent works exploit spatial and temporal information for MOT, but how to combine appearance and temporal features is still not well addressed. In this paper, we propose an innovative and effective tracking method called TrackletNet Tracker (TNT) that combines temporal and appearance information together as a unified framework. First, we define a graph model which treats each tracklet as a vertex. The tracklets are generated by associating detection results frame by frame with the help of the appearance similarity and the spatial consistency. To compensate camera movement, epipolar constraints are taken into consideration in the association. Then, for every pair of two tracklets, the similarity, called the connectivity in the paper, is measured by our designed multi-scale TrackletNet. Afterwards, the tracklets are clustered into groups and each group represents a unique object ID. Our proposed TNT has the ability to handle most of the challenges in MOT, and achieves promising results on MOT16 and MOT17 benchmark datasets compared with other state-of-the-art methods.
MAT: Motion-Aware Multi-Object Tracking

Shoudong Han,Piao Huang,Hongwei Wang,En Yu,Donghaisheng Liu,Xiaofeng Pan,Jun Zhao

DOI: https://doi.org/10.48550/arXiv.2009.04794

2020-09-18

Abstract:Modern multi-object tracking (MOT) systems usually model the trajectories by associating per-frame detections. However, when camera motion, fast motion, and occlusion challenges occur, it is difficult to ensure long-range tracking or even the tracklet purity, especially for small objects. Although re-identification is often employed, due to noisy partial-detections, similar appearance, and lack of temporal-spatial constraints, it is not only unreliable and time-consuming, but still cannot address the false negatives for occluded and blurred objects. In this paper, we propose an enhanced MOT paradigm, namely Motion-Aware Tracker (MAT), focusing more on various motion patterns of different objects. The rigid camera motion and nonrigid pedestrian motion are blended compatibly to form the integrated motion localization module. Meanwhile, we introduce the dynamic reconnection context module, which aims to balance the robustness of long-range motion-based reconnection, and includes the cyclic pseudo-observation updating strategy to smoothly fill in the tracking fragments caused by occlusion or blur. Additionally, the 3D integral image module is presented to efficiently cut useless track-detection association connections with temporal-spatial constraints. Extensive experiments on MOT16 and MOT17 challenging benchmarks demonstrate that our MAT approach can achieve the superior performance by a large margin with high efficiency, in contrast to other state-of-the-art trackers.

Computer Vision and Pattern Recognition
Exploit the Connectivity

Gaoang Wang,Yizhou Wang,Haotian Zhang,Renshu Gu,Jenq-Neng Hwang

DOI: https://doi.org/10.1145/3343031.3350853

2019-01-01

Abstract:Multi-object tracking (MOT) is an important topic and critical task related to both static and moving camera applications, such as traffic flow analysis, autonomous driving and robotic vision. However, due to unreliable detection, occlusion and fast camera motion, tracked targets can be easily lost, which makes MOT very challenging. Most recent works exploit spatial and temporal information for MOT, but how to combine appearance and temporal features is still not well addressed. In this paper, we propose an innovative and effective tracking method called TrackletNet Tracker (TNT) that combines temporal and appearance information together as a unified framework. First, we define a graph model which treats each tracklet as a vertex. The tracklets are generated by associating detection results frame by frame with the help of the appearance similarity and the spatial consistency. To compensate camera movement, epipolar constraints are taken into consideration in the association. Then, for every pair of two tracklets, the similarity, called the connectivity in the paper, is measured by our designed multi-scale TrackletNet. Afterwards, the tracklets are clustered into groups and each group represents a unique object ID. Our proposed TNT has the ability to handle most of the challenges in MOT, and achieves promising results on MOT16 and MOT17 benchmark datasets compared with other state-of-the-art methods.
Split and Connect: A Universal Tracklet Booster for Multi-Object Tracking

Gaoang Wang,Yizhou Wang,Renshu Gu,Weijie Hu,Jenq-Neng Hwang

DOI: https://doi.org/10.1109/tmm.2022.3140919

IF: 7.3

2022-01-01

IEEE Transactions on Multimedia

Abstract:Multi-object tracking (MOT) is an essential task in the computer vision field. With the fast development of deep learning technology in recent years, MOT has achieved great improvement. However, some challenges still remain, such as sensitiveness to occlusion, instability under different lighting conditions, non-robustness to deformable objects, etc. To address such common challenges in most of the existing trackers, in this paper, a tracklet booster algorithm is proposed, which can be built upon any other tracker. The motivation is simple and straightforward: split tracklets on potential ID-switch positions and then connect multiple tracklets into one if they are from the same object. In other words, the tracklet booster consists of two parts, i.e., Splitter and Connector. First, an architecture with stacked temporal dilated convolution blocks is employed for the splitting position prediction via label smoothing strategy with adaptive Gaussian kernels. Then, a multi-head self-attention based encoder is exploited for the tracklet embedding, which is further used to connect tracklets into larger groups. We conduct sufficient experiments on MOT17 and MOT20 benchmark datasets, which demonstrates promising results. Combined with the proposed tracklet booster, existing trackers usually can achieve large improvements on the IDF1 score, which shows the effectiveness of the proposed method.
STMT: Spatio-temporal memory transformer for multi-object tracking

Songbo Gu,Jianxin Ma,Guancheng Hui,Qiyang Xiao,Wentao Shi

DOI: https://doi.org/10.1007/s10489-023-04617-1

IF: 5.3

2023-07-10

Applied Intelligence

Abstract:Typically, modern online Multi-Object Tracking (MOT) methods first obtain the detected objects in each frame and then establish associations between them in successive frames. However, it is difficult to obtain high-quality trajectories when camera motion, fast motion, and occlusion challenges occur. To address these problems, this paper proposes a transformer-based MOT system named Spatio-Temporal Memory Transformer (STMT), which focuses on time and history information. The proposed STMT consists of a Spatio-Temporal Enhancement Module (STEM) that uses 3D convolution to model the spatial and temporal interactions of objects and obtains rich features in spatio-temporal information. Moreover, a Dynamic Spatio-Temporal Memory (DSTM) is presented to associate detections with tracklets and contains three units: an Identity Aggregation Module (IAM), a Linear Dynamic Encoder (LD-Encoder) and a memory Decoder (Decoder). The IAM utilizes the geometric changes of objects to reduce the impact of deformation on tracking performance, the LD-Encoder is used to obtain the dependency between objects, and the Decoder generates appearance similarity scores. Furthermore, a Score Fusion Equilibrium Strategy (SFES) is employed to balance the similarity and position distance fusion scores. Extensive experiments demonstrate that the proposed STMT approach is generally superior to the state-of-the-art trackers on the MOT16 and MOT17 benchmarks.

computer science, artificial intelligence
Multi-object tracking algorithm based on interactive attention network and adaptive trajectory reconnection

Sugang Ma,Shuaipeng Duan,Zhiqiang Hou,Wangsheng Yu,Lei Pu,Xiangmo Zhao

DOI: https://doi.org/10.1016/j.eswa.2024.123581

IF: 8.5

2024-03-06

Expert Systems with Applications

Abstract:Multi-object tracking (MOT) detects multiple targets in an image and assigns a unique identifier to each target. However, challenges such as rapid motion, occlusion, and camera motion in the tracking scene may lead to identity switches (IDs) and missing trajectory problems, which degrade the performance of the tracker. To address these issues, this paper presents an MOT algorithm based on an interactive attention network and adaptive trajectory reconnection. First, an interactive attention network was created to learn the features for two different tasks of detection and tracking to alleviate feature conflicts in order to extract sufficient feature information. A new cost matrix was then designed to fuse the motion and feature information, thereby reducing the number of IDs. Meanwhile, the extreme gradient boosting reconnection module was used to achieve adaptive trajectory reconnection and reduce missing trajectories. The proposed algorithm achieved 61.5 % and 55.4 % HOTA using the standard MOT17 and MOT20 datasets, respectively. In comparison to FairMOT, our algorithm showcased notable enhancements of 3% and 1.6% on these datasets. Furthermore, when compared to state-of-the-art algorithms, the proposed algorithm demonstrated superior tracking performance.

computer science, artificial intelligence,engineering, electrical & electronic,operations research & management science
Online Multi-Object Tracking Using CNN-based Single Object Tracker with Spatial-Temporal Attention Mechanism

Qi Chu,Wanli Ouyang,Hongsheng Li,Xiaogang Wang,Bin Liu,Nenghai Yu

DOI: https://doi.org/10.48550/arXiv.1708.02843

2017-08-14

Abstract:In this paper, we propose a CNN-based framework for online MOT. This framework utilizes the merits of single object trackers in adapting appearance models and searching for target in the next frame. Simply applying single object tracker for MOT will encounter the problem in computational efficiency and drifted results caused by occlusion. Our framework achieves computational efficiency by sharing features and using ROI-Pooling to obtain individual features for each target. Some online learned target-specific CNN layers are used for adapting the appearance model for each target. In the framework, we introduce spatial-temporal attention mechanism (STAM) to handle the drift caused by occlusion and interaction among targets. The visibility map of the target is learned and used for inferring the spatial attention map. The spatial attention map is then applied to weight the features. Besides, the occlusion status can be estimated from the visibility map, which controls the online updating process via weighted loss on training samples with different occlusion statuses in different frames. It can be considered as temporal attention mechanism. The proposed algorithm achieves 34.3% and 46.0% in MOTA on challenging MOT15 and MOT16 benchmark dataset respectively.

Computer Vision and Pattern Recognition
Chained-Tracker: Chaining Paired Attentive Regression Results for End-to-End Joint Multiple-Object Detection and Tracking

Jinlong Peng,Changan Wang,Fangbin Wan,Yang Wu,Yabiao Wang,Ying Tai,Chengjie Wang,Jilin Li,Feiyue Huang,Yanwei Fu

DOI: https://doi.org/10.1007/978-3-030-58548-8_9

2020-01-01

Abstract:Existing Multiple-Object Tracking (MOT) methods either follow the tracking-by-detection paradigm to conduct object detection, feature extraction and data association separately, or have two of the three subtasks integrated to form a partially end-to-end solution. Going beyond these sub-optimal frameworks, we propose a simple online model named Chained-Tracker (CTracker), which naturally integrates all the three subtasks into an end-to-end solution (the first as far as we know). It chains paired bounding boxes regression results estimated from overlapping nodes, of which each node covers two adjacent frames. The paired regression is made attentive by object-attention (brought by a detection module) and identity-attention (ensured by an ID verification module). The two major novelties: chained structure and paired attentive regression, make CTracker simple, fast and effective, setting new MOTA records on MOT16 and MOT17 challenge datasets (67.6 and 66.6, respectively), without relying on any extra training data. The source code of CTracker can be found at: github.com/pjl1995/CTracker .
MeMOTR: Long-Term Memory-Augmented Transformer for Multi-Object Tracking

Ruopeng Gao,Limin Wang

DOI: https://doi.org/10.48550/arXiv.2307.15700

2023-07-28

Computer Vision and Pattern Recognition

Abstract:As a video task, Multiple Object Tracking (MOT) is expected to capture temporal information of targets effectively. Unfortunately, most existing methods only explicitly exploit the object features between adjacent frames, while lacking the capacity to model long-term temporal information. In this paper, we propose MeMOTR, a long-term memory-augmented Transformer for multi-object tracking. Our method is able to make the same object's track embedding more stable and distinguishable by leveraging long-term memory injection with a customized memory-attention layer. This significantly improves the target association ability of our model. Experimental results on DanceTrack show that MeMOTR impressively surpasses the state-of-the-art method by 7.9% and 13.0% on HOTA and AssA metrics, respectively. Furthermore, our model also outperforms other Transformer-based methods on association performance on MOT17 and generalizes well on BDD100K. Code is available at https://github.com/MCG-NJU/MeMOTR.
MotionTrack: Learning Robust Short-term and Long-term Motions for Multi-Object Tracking

Zheng Qin,Sanping Zhou,Le Wang,Jinghai Duan,Gang Hua,Wei Tang

2023-04-17

Abstract:The main challenge of Multi-Object Tracking~(MOT) lies in maintaining a continuous trajectory for each target. Existing methods often learn reliable motion patterns to match the same target between adjacent frames and discriminative appearance features to re-identify the lost targets after a long period. However, the reliability of motion prediction and the discriminability of appearances can be easily hurt by dense crowds and extreme occlusions in the tracking process. In this paper, we propose a simple yet effective multi-object tracker, i.e., MotionTrack, which learns robust short-term and long-term motions in a unified framework to associate trajectories from a short to long range. For dense crowds, we design a novel Interaction Module to learn interaction-aware motions from short-term trajectories, which can estimate the complex movement of each target. For extreme occlusions, we build a novel Refind Module to learn reliable long-term motions from the target's history trajectory, which can link the interrupted trajectory with its corresponding detection. Our Interaction Module and Refind Module are embedded in the well-known tracking-by-detection paradigm, which can work in tandem to maintain superior performance. Extensive experimental results on MOT17 and MOT20 datasets demonstrate the superiority of our approach in challenging scenarios, and it achieves state-of-the-art performances at various MOT metrics.

Computer Vision and Pattern Recognition
CSCMOT: Multi-object tracking based on channel spatial cooperative attention mechanism

Fei Wang,Hao Yan,Libo Zhang,Ke Gao

DOI: https://doi.org/10.1016/j.engappai.2023.107084

IF: 8

2023-09-17

Engineering Applications of Artificial Intelligence

Abstract:Multi-object tracking has made good progress in recent years. Most mainstream methods use the fusion method of detection and Re-ID to complete the multi-target tracking technology. However, the current multi-tracking algorithm is slow and cannot meet the real-time requirements, which makes it difficult to implement in actual scenarios. In addition, the current mainstream multi-target tracking technology often has the problem of identity information jumping. Such frequent identity information hopping can lead to serious problems in some demanding practical applications, resulting in poor tracking performance. To solve these problems, we propose a simple framework CSCMOT. A non-parametric attention mechanism is adopted to focus on some feature points of the target without increasing the amount of computation, so as to reduce the amount of computation and improve the real-time performance of the algorithm. In addition, the jumping problem of identity information can be reduced by random simulation occlusion to improve tracking performance. Experiments show that the real-time performance of the proposed CSCMOT framework reaches 32.5 FPS, which exceeds most of the mainstream methods. In addition, the ID-switch was reduced to 2493 on the MOT17 dataset. Made a great breakthrough, better to solve the problem of identity information jump. The tracking accuracy is also 71.5, a competitive result that exceeds most of the mainstream methods. Effective data show that the framework improves the real-time performance of the algorithm, solves the problem of identity jump between targets, and is more conducive to experiment landing, which is easy to combine with the mobile robot platform.

automation & control systems,computer science, artificial intelligence,engineering, electrical & electronic, multidisciplinary
Joint Spatial-Temporal and Appearance Modeling with Transformer for Multiple Object Tracking

Peng Dai,Yiqiang Feng,Renliang Weng,Changshui Zhang

DOI: https://doi.org/10.48550/arxiv.2205.15495

2022-01-01

Abstract: The recent trend in multiple object tracking (MOT) is heading towards leveraging deep learning to boost the tracking performance. In this paper, we propose a novel solution named TransSTAM, which leverages Transformer to effectively model both the appearance features of each object and the spatial-temporal relationships among objects. TransSTAM consists of two major parts: (1) The encoder utilizes the powerful self-attention mechanism of Transformer to learn discriminative features for each tracklet; (2) The decoder adopts the standard cross-attention mechanism to model the affinities between the tracklets and the detections by taking both spatial-temporal and appearance features into account. TransSTAM has two major advantages: (1) It is solely based on the encoder-decoder architecture and enjoys a compact network design, hence being computationally efficient; (2) It can effectively learn spatial-temporal and appearance features within one model, hence achieving better tracking accuracy. The proposed method is evaluated on multiple public benchmarks including MOT16, MOT17, and MOT20, and it achieves a clear performance improvement in both IDF1 and HOTA with respect to previous state-of-the-art approaches on all the benchmarks. Our code is available at \url{https://github.com/icicle4/TranSTAM}.
Spatial-Semantic and Temporal Attention Mechanism-Based Online Multi-Object Tracking

Fanjie Meng,Xinqing Wang,Dong Wang,Faming Shao,Lei Fu

DOI: https://doi.org/10.3390/s20061653

2020-03-16

Abstract:Multi-object tracking (MOT) plays a crucial role in various platforms. Occlusion and insertion among targets, complex backgrounds and higher real-time requirements increase the difficulty of MOT problems. Most state-of-the-art MOT approaches adopt the tracking-by-detection strategy, which relies on compute-intensive sliding windows or anchoring schemes to detect matching targets or candidates in each frame. In this work, we introduce a more efficient and effective spatial-temporal attention scheme to track multiple objects in various scenarios. Using a semantic-feature-based spatial attention mechanism and a novel Motion Model, we address the insertion and location of candidates. Some online-learned target-specific convolutional neural networks (CNNs) were used to estimate target occlusion and classify by adapting the appearance model. A temporal attention mechanism was adopted to update the online module by balancing current and history frames. Extensive experiments were performed on Karlsruhe Institute of Technologyand Toyota Technological Institute (KITTI) benchmarks and an Armored Target Tracking Dataset (ATTD) built for ground-armored targets. Experimental results show that the proposed method achieved outstanding tracking performance and met the actual application requirements.
TransMOT: Spatial-Temporal Graph Transformer for Multiple Object Tracking

Peng Chu,Jiang Wang,Quanzeng You,Haibin Ling,Zicheng Liu

DOI: https://doi.org/10.48550/arXiv.2104.00194

2021-04-01

Computer Vision and Pattern Recognition

Abstract:Tracking multiple objects in videos relies on modeling the spatial-temporal interactions of the objects. In this paper, we propose a solution named TransMOT, which leverages powerful graph transformers to efficiently model the spatial and temporal interactions among the objects. TransMOT effectively models the interactions of a large number of objects by arranging the trajectories of the tracked objects as a set of sparse weighted graphs, and constructing a spatial graph transformer encoder layer, a temporal transformer encoder layer, and a spatial graph transformer decoder layer based on the graphs. TransMOT is not only more computationally efficient than the traditional Transformer, but it also achieves better tracking accuracy. To further improve the tracking speed and accuracy, we propose a cascade association framework to handle low-score detections and long-term occlusions that require large computational resources to model in TransMOT. The proposed method is evaluated on multiple benchmark datasets including MOT15, MOT16, MOT17, and MOT20, and it achieves state-of-the-art performance on all the datasets.
STCMOT: Spatio-Temporal Cohesion Learning for UAV-Based Multiple Object Tracking

Jianbo Ma,Chuanming Tang,Fei Wu,Can Zhao,Jianlin Zhang,Zhiyong Xu

2024-09-17

Abstract:Multiple object tracking (MOT) in Unmanned Aerial Vehicle (UAV) videos is important for diverse applications in computer vision. Current MOT trackers rely on accurate object detection results and precise matching of target reidentification (ReID). These methods focus on optimizing target spatial attributes while overlooking temporal cues in modelling object relationships, especially for challenging tracking conditions such as object deformation and blurring, etc. To address the above-mentioned issues, we propose a novel Spatio-Temporal Cohesion Multiple Object Tracking framework (STCMOT), which utilizes historical embedding features to model the representation of ReID and detection features in a sequential order. Concretely, a temporal embedding boosting module is introduced to enhance the discriminability of individual embedding based on adjacent frame cooperation. While the trajectory embedding is then propagated by a temporal detection refinement module to mine salient target locations in the temporal field. Extensive experiments on the VisDrone2019 and UAVDT datasets demonstrate our STCMOT sets a new state-of-the-art performance in MOTA and IDF1 metrics. The source codes are released at <a class="link-external link-https" href="https://github.com/ydhcg-BoBo/STCMOT" rel="external noopener nofollow">this https URL</a>.

Computer Vision and Pattern Recognition
Multi-object tracking via deep feature fusion and association analysis

Hui Li,Yapeng Liu,Xiaoguo Liang,Yongfeng Yuan,Yuanzhi Cheng,Guanglei Zhang,Shinichi Tamura

DOI: https://doi.org/10.1016/j.engappai.2023.106527

IF: 8

2023-06-13

Engineering Applications of Artificial Intelligence

Abstract:We describe a tracking-by-detection framework for multi-object tracking (MOT). It first detects the objects of interest in each frame of the video, followed by identifying association with the object detected in the previous frame. A deep association network is described to perform object feature matching in the arbitrary two frames to infer association degree of objects, and then similarity matrix loss is used to calculate association between each object in different frames to achieve an accurate tracking. The novelty of the work lies in the design of a multi-scale fusion strategy by gradually concatenating sub-networks of low-resolution feature maps in parallel to the main network of high-resolution feature maps, in the construction of a deeper backbone network which can enhance the semantic information of object features, and in the use of a siamese network for training a pair of discontinuous frames. The main advantage of our framework is that it avoids missing detection and partial detection. It is particularly suitable for solving the problem of object ID switch caused by occlusion, entering and leaving of objects. Our method is evaluated and demonstrated on the publicly available MOT15, MOT16, MOT17 and MOT20 benchmark datasets. Compared with the state-of-the-art methods, our method achieves better tracking performance, and is therefore, more suited for MOT tasks.

automation & control systems,computer science, artificial intelligence,engineering, electrical & electronic, multidisciplinary
MOT FCG++: Enhanced Representation of Spatio-temporal Motion and Appearance Features

Yanzhao Fang

2024-11-21

Abstract:The goal of multi-object tracking (MOT) is to detect and track all objects in a scene across frames, while maintaining a unique identity for each object. Most existing methods rely on the spatial-temporal motion features and appearance embedding features of the detected objects in consecutive frames. Effectively and robustly representing the spatial and appearance features of long trajectories has become a critical factor affecting the performance of MOT. We propose a novel approach for appearance and spatial-temporal motion feature representation, improving upon the hierarchical clustering association method MOT FCG. For spatialtemporal motion features, we first propose Diagonal Modulated GIoU, which more accurately represents the relationship between the position and shape of the objects. Second, Mean Constant Velocity Modeling is proposed to reduce the effect of observation noise on target motion state estimation. For appearance features, we utilize a dynamic appearance representation that incorporates confidence information, enabling the trajectory appearance features to be more robust and global. Based on the baseline model MOT FCG, we have realized further improvements in the performance of all. we achieved 63.1 HOTA, 76.9 MOTA and 78.2 IDF1 on the MOT17 test set, and also achieved competitive performance on the MOT20 and DanceTrack sets.

Computer Vision and Pattern Recognition,Artificial Intelligence
Multi-object tracking with adaptive measurement noise and information fusion

Xi Huang,Yinwei Zhan

DOI: https://doi.org/10.1016/j.imavis.2024.104964

IF: 3.86

2024-02-28

Image and Vision Computing

Abstract:Multi-object tracking (MOT) is a challenging task in computer vision that aims to estimate the trajectories of multiple objects in a video sequence. Observation-Centric SORT (OCSORT) is a pure motion-based MOT algorithm that uses the Kalman filter as the motion model and three observation-centric techniques: Re-Update, Momentum and Recovery, to enhance the data association. However, OCSORT is limited by camera motion error, constant measurement noise and lack of appearance information. In this paper, we propose three methods to address these limitations and improve the performance of OCSORT. First, we use Enhanced Correlation Coefficient Maximization (ECC) to compensate for the camera motion between adjacent frames. Second, we adjust the measurement noise scale for the Kalman filter according to the detection confidence. Third, we introduce a deep visual feature model to extract appearance information and propose a method to effectively use both motion and appearance information. The proposed method first filters out the inappropriate appearance information based on motion information and then combines the filtered appearance information with the motion information by minimization. We evaluate our algorithm on three MOT benchmarks: MOT17, MOT20 and DanceTrack. The results show that our algorithm achieves state-of-the-art performance on all datasets, especially on DanceTrack, where the objects have highly nonlinear motion and frequent occlusion. Compared to OCSORT, our algorithm improves Higher Order Tracking Accuracy (HOTA) by 1.1%, 0.8%, and 3.5%, and ID F1 Score (IDF1) by 1.7%, 1.9%, and 4.3% on MOT17, MOT20 and DanceTrack, respectively.

computer science, artificial intelligence, theory & methods,engineering, electrical & electronic, software engineering,optics
PSMOT: Online Occlusion-Aware Multi-Object Tracking Exploiting Position Sensitivity

Ranyang Zhao,Xinyan Zhang,Jianwei Zhang

DOI: https://doi.org/10.3390/s24041199

IF: 3.9

2024-02-13

Sensors

Abstract:Models based on joint detection and re-identification (ReID), which significantly increase the efficiency of online multi-object tracking (MOT) systems, are an evolution from separate detection and ReID models in the tracking-by-detection (TBD) paradigm. It is observed that these joint models are typically one-stage, while the two-stage models become obsolete because of their slow speed and low efficiency. However, the two-stage models have naive advantages over the one-stage anchor-based and anchor-free models in handling feature misalignment and occlusion, which suggests that the two-stage models, via meticulous design, could be on par with the state-of-the-art one-stage models. Following this intuition, we propose a robust and efficient two-stage joint model based on R–FCN, whose backbone and neck are fully convolutional, and the RoI-wise process only involves simple calculations. In the first stage, an adaptive sparse anchoring scheme is utilized to produce adequate, high-quality proposals to improve efficiency. To boost both detection and ReID, two key elements—feature aggregation and feature disentanglement—are taken into account. To improve robustness against occlusion, the position-sensitivity is exploited, first to estimate occlusion and then to direct the post-process for anti-occlusion. Finally, we link the model to a hierarchical association algorithm to form a complete MOT system called PSMOT. Compared to other cutting-edge systems, PSMOT achieves competitive performance while maintaining time efficiency.

engineering, electrical & electronic,chemistry, analytical,instruments & instrumentation

[Significance of cardiovascular research within the scope of the total development of medical sciences in East Germany].

Transformer Network for Multi-Person Tracking and Re-Identification in Unconstrained Environment

Exploit the Connectivity: Multi-Object Tracking with TrackletNet

MAT: Motion-Aware Multi-Object Tracking

Exploit the Connectivity

Split and Connect: A Universal Tracklet Booster for Multi-Object Tracking

STMT: Spatio-temporal memory transformer for multi-object tracking

Multi-object tracking algorithm based on interactive attention network and adaptive trajectory reconnection

Online Multi-Object Tracking Using CNN-based Single Object Tracker with Spatial-Temporal Attention Mechanism

Chained-Tracker: Chaining Paired Attentive Regression Results for End-to-End Joint Multiple-Object Detection and Tracking

MeMOTR: Long-Term Memory-Augmented Transformer for Multi-Object Tracking

MotionTrack: Learning Robust Short-term and Long-term Motions for Multi-Object Tracking

CSCMOT: Multi-object tracking based on channel spatial cooperative attention mechanism

Joint Spatial-Temporal and Appearance Modeling with Transformer for Multiple Object Tracking

Spatial-Semantic and Temporal Attention Mechanism-Based Online Multi-Object Tracking

TransMOT: Spatial-Temporal Graph Transformer for Multiple Object Tracking

STCMOT: Spatio-Temporal Cohesion Learning for UAV-Based Multiple Object Tracking

Multi-object tracking via deep feature fusion and association analysis

MOT FCG++: Enhanced Representation of Spatio-temporal Motion and Appearance Features

Multi-object tracking with adaptive measurement noise and information fusion

PSMOT: Online Occlusion-Aware Multi-Object Tracking Exploiting Position Sensitivity