Abstract:Hyperspectral videos can provide the spatial, spectral, and motion information of targets, which makes it possible to track camouflaged targets that are similar to the background. However, hyperspectral object tracking is a challenging task, due to the huge hyperspectral video data dimension and the “data hungry” problem for the model training. Insufficient training data can seriously interfere with the accuracy and generalization of the tracking models. In this paper, a dual deep Siamese network framework for hyperspectral object tracking (SiamHYPER) is proposed for learning a hyperspectral tracker from a pretrained RGB tracker in the case of the “data hungry” problem. Specifically, in addition to a pretrained RGB-based Siamese tracker, a hyperspectral target-aware module is designed to mine the spectral information during the target prediction, and a spatial-spectral cross-attention module is introduced to further fuse the deep spatial and spectral features extracted from the RGB tracker and the hyperspectral target-aware module. Benefiting from the guidance training of the RGB tracker, a robust hyperspectral object tracker can be trained effectively with only a small number of hyperspectral video samples, to overcome the “data hungry” problem. In the experiments conducted in this study, the SiamHYPER framework was verified using SiamBAN and SiamRPN++, with 13 000 frames of hyperspectral videos for training, and achieved the best performance on the publicly available hyperspectral dataset released as part of the WHISPERS Hyperspectral Object Tracking Challenge. The area under the curve (AUC) of SiamHYPER was increased by nearly 8.9% and 7.2%, respectively, when compared with the current state-of-the-art RGB-based and hyperspectral trackers. In addition, the processing speed of SiamHYPER was 19 FPS, which is much higher than that of the current state-of-the-art hyperspectral trackers. The source code is available at zhenliuzhenqi/HOT: Hyperspectral object tracking (github.com).

HRSiam: High-Resolution Siamese Network, Towards Space-Borne Satellite Video Tracking.

SiamOHOT: A Lightweight Dual Siamese Network for Onboard Hyperspectral Object Tracking Via Joint Spatial-Spectral Knowledge Distillation

SiamHYPER: Learning a Hyperspectral Object Tracker from an RGB-Based Tracker

SiamMDM: An Adaptive Fusion Network With Dynamic Template for Real-Time Satellite Video Single Object Tracking

Single Object Tracking in Satellite Videos Based on Feature Enhancement and Multi-Level Matching Strategy

Deep Siamese Network With Motion Fitting for Object Tracking in Satellite Videos

High-resolution satellite video single object tracking based on thicksiam framework

Siamese Multi-Scale Adaptive Search Network for Remote Sensing Single-Object Tracking

Single Object Tracking in Satellite Videos: Deep Siamese Network Incorporating an Interframe Difference Centroid Inertia Motion Model

Pasiam: Predicting Attention Inspired Siamese Network, For Space-Borne Satellite Video Tracking

Online Semantic Subspace Learning with Siamese Network for UAV Tracking

A Location-Aware Siamese Network for High-Speed Visual Tracking

Region-based High-resolution Siamese Network for Robust Visual Tracking

SiamHSFT: A Siamese Network-Based Tracker with Hierarchical Sparse Fusion and Transformer for UAV Tracking.

Online Background Discriminative Learning for Satellite Video Object Tracking

A Siamese Network-Based Tracking Framework for Hyperspectral Video

MobileTrack: Siamese Efficient Mobile Network for High‐speed UAV Tracking

Siamese Dense Pixel-Level Fusion Network for Real-Time UAV Tracking

Satellite Videos Object Tracking Based on Enhanced Correlation Filter with Motion Prediction Network

A Siamese-Detection Network for Real-Time Object Tracking

Real-time object tracking in the wild with Siamese network