Abstract:Hyperspectral videos can provide the spatial, spectral, and motion information of targets, which makes it possible to track camouflaged targets that are similar to the background. However, hyperspectral object tracking is a challenging task, due to the huge hyperspectral video data dimension and the “data hungry” problem for the model training. Insufficient training data can seriously interfere with the accuracy and generalization of the tracking models. In this paper, a dual deep Siamese network framework for hyperspectral object tracking (SiamHYPER) is proposed for learning a hyperspectral tracker from a pretrained RGB tracker in the case of the “data hungry” problem. Specifically, in addition to a pretrained RGB-based Siamese tracker, a hyperspectral target-aware module is designed to mine the spectral information during the target prediction, and a spatial-spectral cross-attention module is introduced to further fuse the deep spatial and spectral features extracted from the RGB tracker and the hyperspectral target-aware module. Benefiting from the guidance training of the RGB tracker, a robust hyperspectral object tracker can be trained effectively with only a small number of hyperspectral video samples, to overcome the “data hungry” problem. In the experiments conducted in this study, the SiamHYPER framework was verified using SiamBAN and SiamRPN++, with 13 000 frames of hyperspectral videos for training, and achieved the best performance on the publicly available hyperspectral dataset released as part of the WHISPERS Hyperspectral Object Tracking Challenge. The area under the curve (AUC) of SiamHYPER was increased by nearly 8.9% and 7.2%, respectively, when compared with the current state-of-the-art RGB-based and hyperspectral trackers. In addition, the processing speed of SiamHYPER was 19 FPS, which is much higher than that of the current state-of-the-art hyperspectral trackers. The source code is available at zhenliuzhenqi/HOT: Hyperspectral object tracking (github.com).

SiamATTRPN: Enhance Visual Tracking with Channel and Spatial Attention

PointSiamRCNN: Target-aware Voxel-based Siamese Tracker for Point Clouds

Background-aware Siamese Network Tracking Based on Salient Feature Fusion

Siamese Refine Polar Mask Prediction Network for Visual Tracking

SiamHYPER: Learning a Hyperspectral Object Tracker from an RGB-Based Tracker

Deformable Siamese Attention Networks for Visual Object Tracking

Residual Attention SiameseRPN for Visual Tracking

Siamese Attentional Cascade Keypoints Network for Visual Object Tracking

Siamese Centerness Prediction Network for Real-Time Visual Object Tracking

Siamese Tracking Network with Spatial-Semantic-Aware Attention and Flexible Spatiotemporal Constraint

SiamHAS: Siamese Tracker with Hierarchical Attention Strategy for Aerial Tracking

Object Tracking Algorithm Based on Channel-interconnection-spatial Attention Mechanism and Siamese Region Proposal Network

SiamST: Siamese Network with Spatio-Temporal Awareness for Object Tracking

Siamese Tracking Network with Multi-attention Mechanism

Visual Tracking With Siamese Network Based on Fast Attention Network

Siamese anchor-free object tracking with multiscale spatial attentions

SiamAPN++: Siamese Attentional Aggregation Network for Real-Time UAV Tracking

Evolution of Siamese Visual Tracking with Slot Attention

SiamATL: Online Update of Siamese Tracking Network via Attentional Transfer Learning

SiamRAAN: Siamese Residual Attentional Aggregation Network for Visual Object Tracking

Siamese Attentional Keypoint Network for High Performance Visual Tracking