Abstract:In this paper, we propose a novel graph model, called weighted sparse representation regularized graph, to learn a robust object representation using multispectral (RGB and thermal) data for visual tracking. In particular, the tracked object is represented with a graph with image patches as nodes. This graph is dynamically learned from two aspects. First, the graph affinity (i.e., graph structure and edge weights) that indicates the appearance compatibility of two neighboring nodes is optimized based on the weighted sparse representation, in which the modality weight is introduced to leverage RGB and thermal information adaptively. Second, each node weight that indicates how likely it belongs to the foreground is propagated from others along with graph affinity. The optimized patch weights are then imposed on the extracted RGB and thermal features, and the target object is finally located by adopting the structured SVM algorithm. Moreover, we also contribute a comprehensive dataset for RGB-T tracking purpose. Comparing with existing ones, the new dataset has the following advantages: 1) Its size is sufficiently large for large-scale performance evaluation (total frame number: 210K, maximum frames per video pair: 8K). 2) The alignment between RGB-T video pairs is highly accurate, which does not need pre- and post-processing. 3) The occlusion levels are annotated for analyzing the occlusion-sensitive performance of different methods. Extensive experiments on both public and newly created datasets demonstrate the effectiveness of the proposed tracker against several state-of-the-art tracking methods.

Grayscale-Thermal Object Tracking Via Multitask Laplacian Sparse Representation

Real-Time Grayscale-Thermal Tracking via Laplacian Sparse Representation.

Learning Collaborative Sparse Representation for Grayscale-Thermal Tracking.

A Fusion Approach to Grayscale-Thermal Tracking with Cross-Modal Sparse Representation.

Object tracking with 3D LIDAR via multi-task sparse learning

Multi-object Tracking Based on Thermal-Visible Video Sequence Fusion

Visual tracking via robust multi-task multi-feature joint sparse representation

Online Hierarchical Sparse Representation of Multifeature for Robust Object Tracking

Tracking in Multimedia Data Via Robust Reweighted Local Multi-Task Sparse Representation for Transportation Surveillance

Multi-scale Patch-Based Sparse Appearance Model for Robust Object Tracking

Object Tracking via Multi-task Gaussian-Laplacian Regression

Multi-Modal Sparse Tracking by Jointing Timing and Modal Consistency

Learning Collaborative Sparse Correlation Filter for Real-Time Multispectral Object Tracking.

Robust Visual Tracking Via Multitask Sparse Correlation Filters Learning

Tracking via Robust Multi-task Multi-view Joint Sparse Representation

Deep Object Tracking with Multi-Modal Data

Modality-correlation-aware sparse representation for RGB-infrared object tracking

Robust Object Tracking Method via Adaptive Modality Selection

Multi-Level Fusion for Robust RGBT Tracking via Enhanced Thermal Representation

Weighted Sparse Representation Regularized Graph Learning for RGB-T Object Tracking

Visual tracking via robust multitask sparse prototypes