Abstract:Human pose estimation is a critical component in autonomous driving and parking, enhancing safety by predicting human actions. Traditional frame-based cameras and videos are commonly applied, yet, they become less reliable in scenarios under high dynamic range or heavy motion blur. In contrast, event cameras offer a robust solution for navigating these challenging contexts. Predominant methodologies incorporate event cameras into learning frameworks by accumulating events into event frames. However, such methods tend to marginalize the intrinsic asynchronous and high temporal resolution characteristics of events. This disregard leads to a loss in essential temporal dimension data, crucial for safety-critical tasks associated with dynamic human activities. To address this issue and to unlock the 3D potential of event information, we introduce two 3D event representations: the Rasterized Event Point Cloud (RasEPC) and the Decoupled Event Voxel (DEV). The RasEPC collates events within concise temporal slices at identical positions, preserving 3D attributes with statistical cues and markedly mitigating memory and computational demands. Meanwhile, the DEV representation discretizes events into voxels and projects them across three orthogonal planes, utilizing decoupled event attention to retrieve 3D cues from the 2D planes. Furthermore, we develop and release EV-3DPW, a synthetic event-based dataset crafted to facilitate training and quantitative analysis in outdoor scenes. On the public real-world DHP19 dataset, our event point cloud technique excels in real-time mobile predictions, while the decoupled event voxel method achieves the highest accuracy. Experiments reveal our proposed 3D representation methods' superior generalization capacities against traditional RGB images and event frame techniques. Our code and dataset are available at https://github.com/MasterHow/EventPointPose.

6-Dof Pose Relocalization for Event Cameras with Entropy Frame and Attention Networks

Rethinking Human Pose Estimation for Autonomous Driving with 3D Event Representations.

Event-Based Fusion for Motion Deblurring with Cross-modal Attention

A Multi-Scale Recurrent Framework for Motion Segmentation With Event Camera

3D Point-to-Keypoint Voting Network for 6D Pose Estimation

Real-Time 6DOF Pose Relocalization for Event Cameras With Stacked Spatial LSTM Networks

A 6-Dofs Event-Based Camera Relocalization System by CNN-LSTM and Image Denoising

6D Camera Relocalization in Visually Ambiguous Extreme Environments

Delving deeper into convolutional neural networks for camera relocalization.

PoseNet: A Convolutional Network for Real-Time 6-DOF Camera Relocalization

Accurate Rgb Camera Relocalization Using Regression Forest

MC-VEO: A Visual-Event Odometry with Accurate 6-Dof Motion Compensation

A Simple and Effective Point-based Network for Event Camera 6-DOFs Pose Relocalization

Dynamic Subframe Splitting and Spatio-Temporal Motion Entangled Sparse Attention for RGB-E Tracking

Reloc3r: Large-Scale Training of Relative Camera Pose Regression for Generalizable, Fast, and Accurate Visual Localization

Local Supports Global: Deep Camera Relocalization With Sequence Enhancement

6D Dynamic Camera Relocalization from Single Reference Image

Self-Supervised Camera Relocalization with Hierarchical Fern Encoding

EffLoc: Lightweight Vision Transformer for Efficient 6-DOF Camera Relocalization

ACF-Net: Attention Context Fusion Network for 6D Pose Estimation

An End-to-end Learning Framework for Visual Camera Relocalization Using RGB and RGB-D Images