Abstract:Radio frequency (RF) devices such as Wi-Fi transceivers, radio frequency identification tags, and millimeter-wave radars have appeared in large numbers in daily lives. The presence and movement of humans can affect the propagation of RF signals, further, this phenomenon is exploited for human action recognition. Compared to camera solutions, RF approaches exhibit greater resilience to occlusions and lighting conditions, while also raising fewer privacy concerns in indoor scenarios. However, current works have many limitations, including the unavailability of datasets, insufficient training samples, and simple or limited action categories for specific applications, which seriously hinder the growth of RF solutions, presenting a significant obstacle in transitioning RF sensing research from the laboratory to a wide range of everyday life applications. To facilitate the transitioning, in this paper, we introduce and release a large-scale multiple radio frequency dataset, named XRF55, for indoor human action analysis. XRF55 encompasses 42.9K RF samples and 55 action classes of human-object interactions, human-human interactions, fitness, body motions, and human-computer interactions, collected from 39 subjects within 100 days. These actions were meticulously selected from 19 RF sensing papers and 16 video action recognition datasets. Each action is chosen to support various applications with high practical value, such as elderly fall detection, fatigue monitoring, domestic violence detection, etc. Moreover, XRF55 contains 23 RFID tags at 922.38MHz, 9 Wi-Fi links at 5.64GHz, one mmWave radar at 60-64GHz, and one Azure Kinect with RGB+D+IR sensors, covering frequency across decimeter wave, centimeter wave, and millimeter wave. In addition, we apply a mutual learning strategy over XRF55 for the task of action recognition. Unlike simple modality fusion, under mutual learning, three RF modalities are trained collaboratively and then work solely. We find these three RF modalities will promote each other. It is worth mentioning that, with synchronized Kinect, XRF55 also supports the exploration of action detection, action segmentation, pose estimation, human parsing, mesh reconstruction, etc., with RF-only or RF-Vision approaches.

SBRF: A Fine-Grained Radar Signal Generator for Human Sensing

Human Recognition with the Optoelectronic Reservoir-Computing-based Micro-Doppler Radar Signal Processing.

AI-Enhanced 3D RF Representation Using Low-Cost mmWave Radar

Differentiable Radio Frequency Ray Tracing for Millimeter-Wave Sensing

Doppler Vital Signs Detection in the Presence of Large-Scale Random Body Movements

RadSimReal: Bridging the Gap Between Synthetic and Real Data in Radar Object Detection With Simulation

SIMFALL: A Data Generator for RF-Based Fall Detection

RadarFormer: End-to-End Human Perception With Through-Wall Radar and Transformers

RF Vital Sign Sensing under Free Body Movement.

RFGAN: RF-Based Human Synthesis

Talk2Radar: Bridging Natural Language with 4D mmWave Radar for 3D Referring Expression Comprehension

Accurate Human Pose Estimation using RF Signals

RF-Net: a Unified Meta-learning Framework for RF-enabled One-shot Human Activity Recognition

Radar Can See and Hear as Well: A New Multimodal Benchmark Based on Radar Sensing

PRISM: Pre-training RF Signals in Sparsity-aware Masked Autoencoders

mmBody Benchmark: 3D Body Reconstruction Dataset and Analysis for Millimeter Wave Radar

MoRe-Fi: Motion-robust and Fine-grained Respiration Monitoring via Deep-Learning UWB Radar

A CMOS-Integrated Radar-Assisted Cognitive Sensing Platform for Seamless Human-Robot Interactions.

XRF55: A Radio Frequency Dataset for Human Indoor Action Analysis

Motion-Robust Contactless Heartbeat Sensing Using 4-D Imaging Radar

RF-Based Human Activity Recognition Using Signal Adapted Convolutional Neural Network