Abstract:An event camera is a novel bio-inspired sensor that effectively compensates for the shortcomings of current frame cameras, which include high latency, low dynamic range, motion blur, etc. Rather than capturing images at a fixed frame rate, an event camera produces an asynchronous signal by measuring the brightness change of each pixel. Consequently, an appropriate algorithm framework that can handle the unique data types of event-based vision is required. In this paper, we propose a dynamic object tracking framework using an event camera to achieve long-term stable tracking of event objects. One of the key novel features of our approach is to adopt an adaptive strategy that adjusts the spatiotemporal domain of event data. To achieve this, we reconstruct event images from high-speed asynchronous streaming data via online learning. Additionally, we apply the Siamese network to extract features from event data. In contrast to earlier models that only extract hand-crafted features, our method provides powerful feature description and a more flexible reconstruction strategy for event data. We assess our algorithm in three challenging scenarios: 6-DoF (six degrees of freedom), translation, and rotation. Unlike fixed cameras in traditional object tracking tasks, all three tracking scenarios involve the simultaneous violent rotation and shaking of both the camera and objects. Results from extensive experiments suggest that our proposed approach achieves superior accuracy and robustness compared to other state-of-the-art methods. Without reducing time efficiency, our novel method exhibits a 30% increase in accuracy over other recent models. Furthermore, results indicate that event cameras are capable of robust object tracking, which is a task that conventional cameras cannot adequately perform, especially for super-fast motion tracking and challenging lighting situations.

CrossEI: Boosting Motion-Oriented Object Tracking With an Event Camera

Intensity/Inertial Integration-Aided Feature Tracking on Event Cameras

Event-Based Fusion for Motion Deblurring with Cross-modal Attention

A Multi-Scale Recurrent Framework for Motion Segmentation With Event Camera

Object Tracking by Jointly Exploiting Frame and Event Domain

Multi-Stage Fusion for Event-based Multimodal Tracker

EVtracker: An Event-Driven Spatiotemporal Method for Dynamic Object Tracking

A Universal Event-Based Plug-In Module for Visual Object Tracking in Degraded Conditions

VisEvent: Reliable Object Tracking via Collaboration of Frame and Event Flows

Towards Robust Keypoint Detection and Tracking: A Fusion Approach with Event-Aligned Image Features

Enhancing Robustness in Asynchronous Feature Tracking for Event Cameras Through Fusing Frame Steams

MC-VEO: A Visual-Event Odometry with Accurate 6-Dof Motion Compensation

An Event-Driven Asynchronous Feature Tracking Method

TENet: Targetness Entanglement Incorporating with Multi-Scale Pooling and Mutually-Guided Fusion for RGB-E Object Tracking

Cross-Modal Semi-Dense 6-DoF Tracking of an Event Camera in Challenging Conditions

Standard and Event Cameras Fusion for Feature Tracking

Tracking-Assisted Object Detection with Event Cameras

Cross-Modal Semidense 6-DOF Tracking of an Event Camera in Challenging Conditions

Asynchronous Tracking-by-Detection on Adaptive Time Surfaces for Event-based Object Tracking

Frame-based Feature Tracking and EKF Framework for Event Cameras