Abstract:Human pose estimation is a critical component in autonomous driving and parking, enhancing safety by predicting human actions. Traditional frame-based cameras and videos are commonly applied, yet, they become less reliable in scenarios under high dynamic range or heavy motion blur. In contrast, event cameras offer a robust solution for navigating these challenging contexts. Predominant methodologies incorporate event cameras into learning frameworks by accumulating events into event frames. However, such methods tend to marginalize the intrinsic asynchronous and high temporal resolution characteristics of events. This disregard leads to a loss in essential temporal dimension data, crucial for safety-critical tasks associated with dynamic human activities. To address this issue and to unlock the 3D potential of event information, we introduce two 3D event representations: the Rasterized Event Point Cloud (RasEPC) and the Decoupled Event Voxel (DEV). The RasEPC collates events within concise temporal slices at identical positions, preserving 3D attributes with statistical cues and markedly mitigating memory and computational demands. Meanwhile, the DEV representation discretizes events into voxels and projects them across three orthogonal planes, utilizing decoupled event attention to retrieve 3D cues from the 2D planes. Furthermore, we develop and release EV-3DPW, a synthetic event-based dataset crafted to facilitate training and quantitative analysis in outdoor scenes. On the public real-world DHP19 dataset, our event point cloud technique excels in real-time mobile predictions, while the decoupled event voxel method achieves the highest accuracy. Experiments reveal our proposed 3D representation methods' superior generalization capacities against traditional RGB images and event frame techniques. Our code and dataset are available at https://github.com/MasterHow/EventPointPose.

RT-Pose: A 4D Radar Tensor-based 3D Human Pose Estimation and Localization Benchmark

UWB-Radar-Based 3D Human Pose Estimation Using Micro-Range/Micro-Doppler Images

Rethinking Human Pose Estimation for Autonomous Driving with 3D Event Representations.

HuPR: A Benchmark for Human Pose Estimation Using Millimeter Wave Radar

SCRP-Radar: Space-Aware Coordinate Representation for Human Pose Estimation Based on SISO UWB Radar

X-HRNet: Towards Lightweight Human Pose Estimation with Spatially Unidimensional Self-Attention

Indoor 3D Human Pose Estimation Using Single Millimeter-wave Radar and Conditional Generative Adversarial Network

Three-Dimensional Human Pose Estimation from Micro-Doppler Signature Based on SISO UWB Radar

RTMW: Real-Time Multi-Person 2D and 3D Whole-body Pose Estimation

Real-Time Through-Wall Multiperson 3-D Pose Estimation Based on MIMO Radar

HmPEAR: A Dataset for Human Pose Estimation and Action Recognition

A Joint Global–Local Network for Human Pose Estimation With Millimeter Wave Radar

Capturing Human Pose Using Mmwave Radar.

Accurate Human Pose Estimation using RF Signals

Pre-training a Density-Aware Pose Transformer for Robust LiDAR-based 3D Human Pose Estimation

RPM 2.0: RF-Based Pose Machines for Multi-Person 3D Pose Estimation

RTMPose: Real-Time Multi-Person Pose Estimation based on MMPose

LidPose: Real-Time 3D Human Pose Estimation in Sparse Lidar Point Clouds with Non-Repetitive Circular Scanning Pattern

Efficient Human Pose Estimation via 3D Event Point Cloud

LiveHPS: LiDAR-based Scene-level Human Pose and Shape Estimation in Free Environment

Evaluating 3D Human Pose Estimation in Occluded Multi-Sensor Scenarios: Dataset and Annotation Approach