Abstract:Radio-frequency (RF) based human sensing technologies, due to their great practical value in various applications and privacy-preserving nature, have gained tremendous attention in recent years. However, without fully exploiting the characteristics of radio signals, the performance of existing methods are still limited. First, RF features of the moving human body have different representations in dimensions such as channel and scale, which is challenging when performing feature fusion. Besides, the human body is specularly reflective with respect to the radar, which means the human body cannot be fully captured by a single RF snapshot. Therefore, the radar signal reflected by the human body is sparse and incomplete, which is difficult to extract high-quality features for 3D human pose estimation. In this paper, we present the RF-based Pose Machines (RPM), a novel framework which can generate 3D skeletons from RF signals. Considering the characteristics of RF signals, RPM includes several modules to overcome the challenges. Firstly, a Feature Fusion Network (FFN) is designed to effectively fuse radio signals from horizontal and vertical planes based on the channels' correlation and maintain high-quality feature via a multi-scale fusion block. A Spatio-Temporal Attention network is then designed to reconstruct 3D skeletons from the sparse and incomplete RF signals. Specifically, a spatial attention module is designed to model non-local relationships among joints and reconstruct body parts that a single RF snapshot cannot capture. Afterwards, a temporal attention module is proposed to refine 3D pose based on temporal coherency learned from frame queries. To evaluate the performance of our RPM framework, we construct a large-scale dataset of synchronized 3 d skeletons and RF signals, RFSkeleton3D. Our experimental results show that RPM locates 3D key points of the human body with an average error of $5.71\;\mathrm{cm}$ and maintains its performance in new environments with occlusion or bad illumination. The dataset and codes will be made in public.

RFPose-OT: RF-Based 3D Human Pose Estimation via Optimal Transport Theory

Fast 3D Human Pose Estimation Using RF Signals

MobiRFPose: Portable RF-Based 3D Human Pose Camera

UWB-Radar-Based 3D Human Pose Estimation Using Micro-Range/Micro-Doppler Images

Accurate Human Pose Estimation using RF Signals

Motion Imitation of a Humanoid Robot Via Pose Estimation

Mpose: Environment- and Subject-Agnostic 3D Skeleton Posture Reconstruction Leveraging a Single Mmwave Device

RFID-Pose: Vision-Aided Three-Dimensional Human Pose Estimation With Radio-Frequency Identification

RPM: RF-based Pose Machines

RFID-Pose: Vision-aided 3D Human Pose Estimation with RFID

RFPose-GAN: Data Augmentation for RFID Based 3D Human Pose Tracking

RF-based 3D skeletons

Demo Abstract: Vision-aided 3D Human Pose Estimation with RFID.

RF-MvCare: RFID-based 3D Human Pose Estimation for Non-lineof-sight Conditions

RPM 2.0: RF-Based Pose Machines for Multi-Person 3D Pose Estimation

RF-based Multi-view Pose Machine for Multi-Person 3D Pose Estimation.

Indoor 3D Human Pose Estimation Using Single Millimeter-wave Radar and Conditional Generative Adversarial Network

OCHID-Fi: Occlusion-Robust Hand Pose Estimation in 3D via RF-Vision

RF-Care: RFID-based Human Pose Estimation for Nursing-care Applications under Visual Occlusion

From Point to Space: 3D Moving Human Pose Estimation Using Commodity WiFi

RFID-based 3D Human Pose Tracking: A Subject Generalization Approach