Abstract:Human pose estimation is a critical component in autonomous driving and parking, enhancing safety by predicting human actions. Traditional frame-based cameras and videos are commonly applied, yet, they become less reliable in scenarios under high dynamic range or heavy motion blur. In contrast, event cameras offer a robust solution for navigating these challenging contexts. Predominant methodologies incorporate event cameras into learning frameworks by accumulating events into event frames. However, such methods tend to marginalize the intrinsic asynchronous and high temporal resolution characteristics of events. This disregard leads to a loss in essential temporal dimension data, crucial for safety-critical tasks associated with dynamic human activities. To address this issue and to unlock the 3D potential of event information, we introduce two 3D event representations: the Rasterized Event Point Cloud (RasEPC) and the Decoupled Event Voxel (DEV). The RasEPC collates events within concise temporal slices at identical positions, preserving 3D attributes with statistical cues and markedly mitigating memory and computational demands. Meanwhile, the DEV representation discretizes events into voxels and projects them across three orthogonal planes, utilizing decoupled event attention to retrieve 3D cues from the 2D planes. Furthermore, we develop and release EV-3DPW, a synthetic event-based dataset crafted to facilitate training and quantitative analysis in outdoor scenes. On the public real-world DHP19 dataset, our event point cloud technique excels in real-time mobile predictions, while the decoupled event voxel method achieves the highest accuracy. Experiments reveal our proposed 3D representation methods' superior generalization capacities against traditional RGB images and event frame techniques. Our code and dataset are available at https://github.com/MasterHow/EventPointPose.

AgentI2P: Optimizing Image-to-Point Cloud Registration Via Behaviour Cloning and Reinforcement Learning.

Rethinking Human Pose Estimation for Autonomous Driving with 3D Event Representations.

ReAgent: Point Cloud Registration using Imitation and Reinforcement Learning

CMR-Agent: Learning a Cross-Modal Agent for Iterative Image-to-Point Cloud Registration

Mastering Robot Control Through Point-based Reinforcement Learning with Pre-training.

Active search and coverage using point-cloud reinforcement learning

DeepICP: An End-to-End Deep Neural Network for 3D Point Cloud Registration

An Artificial Agent for Robust Image Registration

Active 6D Multi-Object Pose Estimation in Cluttered Scenarios with Deep Reinforcement Learning

Point Cloud Registration via Heuristic Reward Reinforcement Learning

RISE: 3D Perception Makes Real-World Robot Imitation Simple and Effective

Multi-Agent Deep Reinforcement Learning for Online 3D Human Poses Estimation

Efficient Domain Coverage for Vehicles with Second-Order Dynamics via Multi-Agent Reinforcement Learning

PCRMLP: A Two-Stage Network for Point Cloud Registration in Urban Scenes

Point Cloud Matters: Rethinking the Impact of Different Observation Spaces on Robot Learning

A Pose Association Method for Multi-Agent System Based on LiDAR Pointcloud Registration Algorithm

Proactive Multi-Camera Collaboration For 3D Human Pose Estimation

TrackAgent: 6D Object Tracking via Reinforcement Learning

Learning Efficient Multi-Agent Cooperative Visual Exploration

On the Efficacy of 3D Point Cloud Reinforcement Learning

End-to-end 2D-3D Registration between Image and LiDAR Point Cloud for Vehicle Localization