Abstract:A memory mechanism has attracted growing popularity in tracking tasks due to the ability of learning long-term-dependent information. However, it is very challenging for existing memory modules to provide the intrinsic attribute information of the target to the tracker in complex scenes. In this article, by considering the biological visual memory mechanisms, we propose the novel online tracking method via an attention-driven memory network, which can mine discriminative memory information and enhance the robustness and reliability of the tracker. First, to reinforce effectiveness of memory content, we design a novel attention-driven memory network. In the network, the long memory module gains property-level memory information by focusing on the state of the target at both the channel and spatial levels. Meanwhile, in reciprocity, we add a short-term memory module to maintain good adaptability when confronting drastic deformation of the target. The attention-driven memory network can adaptively adjust the contribution of short-term and long-term memories to tracking results under the weighted gradient harmonized loss. On this basis, to avoid model performance degradation, an online memory updater (MU) is further proposed. It is designed to mining for target information in tracking results through the Mixer layer and the online head network together. By evaluating the confidence of the tracking results, the memory updater can accurately judge the time of updating the model, which guarantees the effectiveness of online memory updates. Finally, the proposed method performs favorably and has been extensively validated on several benchmark datasets, including object tracking benchmark-50/100 (OTB-50/100), temple color-128 (TC-128), unmanned aerial vehicles-123 (UAV-123), generic object tracking -10k (GOT-10k), visual object tracking-2016 (VOT-2016), and VOT-2018 against several advanced methods.

An attention selection system based on neural network and its application in tracking objects

Attention Selection with Self-supervised Competition Neural Network and Its Applications in Robot

An Attention Selection Model With Visual Memory And Online Learning

Attention-Driven Memory Network for Online Visual Tracking.

Learning reinforced attentional representation for end-to-end visual tracking

Collaboration of Spatial and Feature Attention for Visual Tracking

Attentional Neural Network: Feature Selection Using Cognitive Feedback

Deep Attentive Tracking via Reciprocative Learning

Design of Adaptive Target Tracking Algorithm for Robots Based on Visual Attention Mechanism

Siamese-Based Attention Learning Networks for Robust Visual Object Tracking

Light-weight UAV object tracking network based on strategy gradient and attention mechanism

A Spatial–Channel–Temporal-Fused Attention for Spiking Neural Networks

A Bayesian Detect to Track System for Robust Visual Object Tracking and Semi-Supervised Model Learning

Utilising Visual Attention Cues for Vehicle Detection and Tracking

Tracking by Joint Local and Global Search: A Target-Aware Attention-Based Approach

Object Based Attention Through Internal Gating

Robust visual tracking based on selective attention shift

Attention-Mechanism-based Tracking Method for Intelligent Internet of Vehicles

Modelling attention control using a convolutional neural network designed after the ventral visual pathway

A brain-inspired object-based attention network for multiobject recognition and visual reasoning

Hebbian-Based Neural Networks for Bottom-Up Visual Attention Systems