Abstract:A memory mechanism has attracted growing popularity in tracking tasks due to the ability of learning long-term-dependent information. However, it is very challenging for existing memory modules to provide the intrinsic attribute information of the target to the tracker in complex scenes. In this article, by considering the biological visual memory mechanisms, we propose the novel online tracking method via an attention-driven memory network, which can mine discriminative memory information and enhance the robustness and reliability of the tracker. First, to reinforce effectiveness of memory content, we design a novel attention-driven memory network. In the network, the long memory module gains property-level memory information by focusing on the state of the target at both the channel and spatial levels. Meanwhile, in reciprocity, we add a short-term memory module to maintain good adaptability when confronting drastic deformation of the target. The attention-driven memory network can adaptively adjust the contribution of short-term and long-term memories to tracking results under the weighted gradient harmonized loss. On this basis, to avoid model performance degradation, an online memory updater (MU) is further proposed. It is designed to mining for target information in tracking results through the Mixer layer and the online head network together. By evaluating the confidence of the tracking results, the memory updater can accurately judge the time of updating the model, which guarantees the effectiveness of online memory updates. Finally, the proposed method performs favorably and has been extensively validated on several benchmark datasets, including object tracking benchmark-50/100 (OTB-50/100), temple color-128 (TC-128), unmanned aerial vehicles-123 (UAV-123), generic object tracking -10k (GOT-10k), visual object tracking-2016 (VOT-2016), and VOT-2018 against several advanced methods.

Aligned Spatial-Temporal Memory Network for Thermal Infrared Target Tracking

RASTMTrack: Robust and Adaptive Space-Time Memory Networks for Visual Tracking

A Fourier-Transform-Based Framework with Asymptotic Attention for Mobile Thermal InfraRed Object Detection

Exploit Spatiotemporal Contextual Information for 3D Single Object Tracking Via Memory Networks

Hierarchical Attention Siamese Network for Thermal Infrared Target Tracking

Object Tracking via Spatial-Temporal Memory Network

Learning Dual-Level Deep Representation for Thermal Infrared Tracking

Hierarchical Spatial-aware Siamese Network for Thermal Infrared Object Tracking

STMTrack: Template-free Visual Tracking with Space-time Memory Networks

Exploring reliable infrared object tracking with spatio-temporal fusion transformer

The Thermal Infrared Visual Object Tracking VOT-TIR2015 Challenge Results

Dynamic memory network with spatial-temporal feature fusion for visual tracking

Memory Network with Pixel-level Spatio-Temporal Learning for Visual Object Tracking

Visual Tracking via Dynamic Memory Networks

4DST-BTMD: an Infrared Small Target Detection Method Based on 4-D Data-Sphered Space

Attention-Driven Memory Network for Online Visual Tracking.

Coordinate-Aware Thermal Infrared Tracking Via Natural Language Modeling

Learning Spatio-Appearance Memory Network for High-Performance Visual Tracking

Two-Stage Spatio-Temporal Feature Correlation Network for Infrared Ground Target Tracking

Real-Time RGBT Target Tracking Based on Attention Mechanism

Twofold Structured Features-Based Siamese Network for Infrared Target Tracking