Abstract:A memory mechanism has attracted growing popularity in tracking tasks due to the ability of learning long-term-dependent information. However, it is very challenging for existing memory modules to provide the intrinsic attribute information of the target to the tracker in complex scenes. In this article, by considering the biological visual memory mechanisms, we propose the novel online tracking method via an attention-driven memory network, which can mine discriminative memory information and enhance the robustness and reliability of the tracker. First, to reinforce effectiveness of memory content, we design a novel attention-driven memory network. In the network, the long memory module gains property-level memory information by focusing on the state of the target at both the channel and spatial levels. Meanwhile, in reciprocity, we add a short-term memory module to maintain good adaptability when confronting drastic deformation of the target. The attention-driven memory network can adaptively adjust the contribution of short-term and long-term memories to tracking results under the weighted gradient harmonized loss. On this basis, to avoid model performance degradation, an online memory updater (MU) is further proposed. It is designed to mining for target information in tracking results through the Mixer layer and the online head network together. By evaluating the confidence of the tracking results, the memory updater can accurately judge the time of updating the model, which guarantees the effectiveness of online memory updates. Finally, the proposed method performs favorably and has been extensively validated on several benchmark datasets, including object tracking benchmark-50/100 (OTB-50/100), temple color-128 (TC-128), unmanned aerial vehicles-123 (UAV-123), generic object tracking -10k (GOT-10k), visual object tracking-2016 (VOT-2016), and VOT-2018 against several advanced methods.

Attention Selection with Self-supervised Competition Neural Network and Its Applications in Robot

An attention selection system based on neural network and its application in tracking objects

An Attention Selection Model With Visual Memory And Online Learning

Attention-Driven Memory Network for Online Visual Tracking.

Better Deep Visual Attention with Reinforcement Learning in Action Recognition.

Vision attention learning model and its application in robot

A Neuro-Cognitive System and Its Application in Robotics

Where to Look: A Unified Attention Model for Visual Recognition with Reinforcement Learning

Self-Selection Salient Region-Based Scene Recognition Using Slight-Weight Convolutional Neural Network

A Spatial–Channel–Temporal-Fused Attention for Spiking Neural Networks

Object detection based on an adaptive attention mechanism

Attention or memory? Neurointerpretable agents in space and time

Modelling attention control using a convolutional neural network designed after the ventral visual pathway

Design of Adaptive Target Tracking Algorithm for Robots Based on Visual Attention Mechanism

Unsupervised Online Learning for Robotic Interestingness With Visual Memory

Salient Positions based Attention Network for Image Classification

Attention Based Network for Remote Sensing Scene Classification.

Learning to attend in a brain-inspired deep neural network

Learning what and where to attend

Switchable Self-attention Module

Selective Visual Attention: Computational Models and Applications