Abstract:An event camera is a novel bio-inspired sensor that effectively compensates for the shortcomings of current frame cameras, which include high latency, low dynamic range, motion blur, etc. Rather than capturing images at a fixed frame rate, an event camera produces an asynchronous signal by measuring the brightness change of each pixel. Consequently, an appropriate algorithm framework that can handle the unique data types of event-based vision is required. In this paper, we propose a dynamic object tracking framework using an event camera to achieve long-term stable tracking of event objects. One of the key novel features of our approach is to adopt an adaptive strategy that adjusts the spatiotemporal domain of event data. To achieve this, we reconstruct event images from high-speed asynchronous streaming data via online learning. Additionally, we apply the Siamese network to extract features from event data. In contrast to earlier models that only extract hand-crafted features, our method provides powerful feature description and a more flexible reconstruction strategy for event data. We assess our algorithm in three challenging scenarios: 6-DoF (six degrees of freedom), translation, and rotation. Unlike fixed cameras in traditional object tracking tasks, all three tracking scenarios involve the simultaneous violent rotation and shaking of both the camera and objects. Results from extensive experiments suggest that our proposed approach achieves superior accuracy and robustness compared to other state-of-the-art methods. Without reducing time efficiency, our novel method exhibits a 30% increase in accuracy over other recent models. Furthermore, results indicate that event cameras are capable of robust object tracking, which is a task that conventional cameras cannot adequately perform, especially for super-fast motion tracking and challenging lighting situations.

Spatiotemporal Feature Learning for Event-Based Vision

Intensity/Inertial Integration-Aided Feature Tracking on Event Cameras

Event Stream Learning Using Spatio-Temporal Event Surface

Multi-scale Harmonic Mean Time Surfaces for Event-based Object Classification

An Event-based Feature Representation Method for Event Stream Classification Using Deep Spiking Neural Networks

ECSNet: Spatio-Temporal Feature Learning for Event Camera

Spatio-Temporal Recurrent Networks for Event-Based Optical Flow Estimation

EVtracker: An Event-Driven Spatiotemporal Method for Dynamic Object Tracking

An Event-based Categorization Model Using Spatio-temporal Features in a Spiking Neural Network.

Spatiotemporal Filtering for Event-Based Action Recognition

Bio-inspired Categorization Using Event-Driven Feature Extraction and Spike-Based Learning.

Data-driven Feature Tracking for Event Cameras

Event-based Object Detection with Lightweight Spatial Attention Mechanism

Event camera object recognition using spatiotemporal event time surface and reward-modulated spike-timing-dependent plasticity learning rule

BlinkTrack: Feature Tracking over 100 FPS via Events and Images

Token-Based Spatiotemporal Representation of the Events.

Event Stream Super-Resolution Via Spatiotemporal Constraint Learning

A Lightweight Spatiotemporal Network for Online Eye Tracking with Event Camera

Path-adaptive Spatio-Temporal State Space Model for Event-based Recognition with Arbitrary Duration

A Universal Event-Based Plug-In Module for Visual Object Tracking in Degraded Conditions

Pose-Invariant Object Recognition for Event-Based Vision with Slow-ELM