Abstract:An event camera is a novel bio-inspired sensor that effectively compensates for the shortcomings of current frame cameras, which include high latency, low dynamic range, motion blur, etc. Rather than capturing images at a fixed frame rate, an event camera produces an asynchronous signal by measuring the brightness change of each pixel. Consequently, an appropriate algorithm framework that can handle the unique data types of event-based vision is required. In this paper, we propose a dynamic object tracking framework using an event camera to achieve long-term stable tracking of event objects. One of the key novel features of our approach is to adopt an adaptive strategy that adjusts the spatiotemporal domain of event data. To achieve this, we reconstruct event images from high-speed asynchronous streaming data via online learning. Additionally, we apply the Siamese network to extract features from event data. In contrast to earlier models that only extract hand-crafted features, our method provides powerful feature description and a more flexible reconstruction strategy for event data. We assess our algorithm in three challenging scenarios: 6-DoF (six degrees of freedom), translation, and rotation. Unlike fixed cameras in traditional object tracking tasks, all three tracking scenarios involve the simultaneous violent rotation and shaking of both the camera and objects. Results from extensive experiments suggest that our proposed approach achieves superior accuracy and robustness compared to other state-of-the-art methods. Without reducing time efficiency, our novel method exhibits a 30% increase in accuracy over other recent models. Furthermore, results indicate that event cameras are capable of robust object tracking, which is a task that conventional cameras cannot adequately perform, especially for super-fast motion tracking and challenging lighting situations.

Event Stream-based Visual Object Tracking: A High-Resolution Benchmark Dataset and A Novel Baseline

VisEvent: Reliable Object Tracking via Collaboration of Frame and Event Flows

Long-term Frame-Event Visual Tracking: Benchmark Dataset and Baseline

TrackingNet: A Large-Scale Dataset and Benchmark for Object Tracking in the Wild

Revisiting Color-Event based Tracking: A Unified Network, Dataset, and Metric

CRSOT: Cross-Resolution Object Tracking using Unaligned Frame and Event Cameras

The Visual Object Tracking VOT2013 Challenge Results

Learning Adaptive Spatio-Temporal Inference Transformer for Coarse-to-Fine Animal Visual Tracking: Algorithm and Benchmark

Visible-Thermal UAV Tracking: A Large-Scale Benchmark and New Baseline

EVtracker: An Event-Driven Spatiotemporal Method for Dynamic Object Tracking

Distractor-Aware Event-Based Tracking

Visual tracking with screening region enrichment and target validation

DIVOTrack: A Novel Dataset and Baseline Method for Cross-View Multi-Object Tracking in DIVerse Open Scenes

BlinkTrack: Feature Tracking over 100 FPS via Events and Images

VastTrack: Vast Category Visual Object Tracking

High-temporal-resolution event-based vehicle detection and tracking

360VOT: A New Benchmark Dataset for Omnidirectional Visual Object Tracking

SDSTrack: Self-Distillation Symmetric Adapter Learning for Multi-Modal Visual Object Tracking

MambaEVT: Event Stream based Visual Object Tracking using State Space Model

Modeling of Multiple Spatial-Temporal Relations for Robust Visual Object Tracking