Abstract:Human pose estimation is a critical component in autonomous driving and parking, enhancing safety by predicting human actions. Traditional frame-based cameras and videos are commonly applied, yet, they become less reliable in scenarios under high dynamic range or heavy motion blur. In contrast, event cameras offer a robust solution for navigating these challenging contexts. Predominant methodologies incorporate event cameras into learning frameworks by accumulating events into event frames. However, such methods tend to marginalize the intrinsic asynchronous and high temporal resolution characteristics of events. This disregard leads to a loss in essential temporal dimension data, crucial for safety-critical tasks associated with dynamic human activities. To address this issue and to unlock the 3D potential of event information, we introduce two 3D event representations: the Rasterized Event Point Cloud (RasEPC) and the Decoupled Event Voxel (DEV). The RasEPC collates events within concise temporal slices at identical positions, preserving 3D attributes with statistical cues and markedly mitigating memory and computational demands. Meanwhile, the DEV representation discretizes events into voxels and projects them across three orthogonal planes, utilizing decoupled event attention to retrieve 3D cues from the 2D planes. Furthermore, we develop and release EV-3DPW, a synthetic event-based dataset crafted to facilitate training and quantitative analysis in outdoor scenes. On the public real-world DHP19 dataset, our event point cloud technique excels in real-time mobile predictions, while the decoupled event voxel method achieves the highest accuracy. Experiments reveal our proposed 3D representation methods' superior generalization capacities against traditional RGB images and event frame techniques. Our code and dataset are available at https://github.com/MasterHow/EventPointPose.

A Time-Related Voxel Representation Method for Event Camera

Rethinking Human Pose Estimation for Autonomous Driving with 3D Event Representations.

Token-Based Spatiotemporal Representation of the Events.

Multi-scale Harmonic Mean Time Surfaces for Event-based Object Classification

Graph-based Asynchronous Event Processing for Rapid Object Recognition

EVtracker: An Event-Driven Spatiotemporal Method for Dynamic Object Tracking

Event camera object recognition using spatiotemporal event time surface and reward-modulated spike-timing-dependent plasticity learning rule

Event Voxel Set Transformer for Spatiotemporal Representation Learning on Event Streams

Backlight and dim space object detection based on a novel event camera

Time-Ordered Recent Event (TORE) Volumes for Event Cameras

Event Transformer

Dense Voxel 3D Reconstruction Using a Monocular Event Camera

VESS - Variable Event Stream Structure for Event-based Instance Segmentation Benchmark.

Rethinking Efficient and Effective Point-based Networks for Event Camera Classification and Regression: EventMamba

Temporal Up-Sampling for Asynchronous Events

Event Vision-based Corner Detection with Count-normalized Multi-Layer Perceptron and Throughput Indicator

VMV-GCN: Volumetric Multi-View Based Graph CNN for Event Stream Classification

Temporal-Mapping Photography for Event Cameras

Continuous-time Object Segmentation using High Temporal Resolution Event Camera

Eventmd: High-Speed Moving Object Detection Based on Event-Based Video Frames

A Cascaded Timestamp-Free Event Camera Image Compression Method for Gesture Recognition