Abstract:Human action recognition using commercial millimeter wave radar is gaining significant attention in smart elderly care and smart homes. Due to privacy concerns, the sensing data often needs to be processed locally on embedded systems with restricted computational resources, necessitating a balance between recognition accuracy and efficiency. In this paper, we propose a fast human action recognition framework based on 3D point cloud sequences generated by commercial 4D millimeter wave imaging radar systems. The framework comprises two primary phases: data preprocessing and spatial-temporal feature extraction. During the data preprocessing phase, we employ a sliding window approach for frame fusion to enhance the spatial information of the sparse point cloud while retaining its temporal features. Additionally, Morton coding is used to address the disorderliness in the point cloud sequence. For spatial-temporal feature extraction, we introduce an innovative two-stage algorithm. In the spatial feature extraction stage, we initially extract local spatial features for each point, utilizing self-attention to construct a local graph and circumvent the limitations of using Euclidean distance in sparse point clouds. Subsequently, 3D frame fusion convolution is applied to extract spatial features at the frame level, reducing the length of the spatial feature map sequence and lowering computational requirements for subsequent temporal feature extraction. In the temporal feature extraction stage, we employ a modified Transformer encoder with fine-grained feature fusion to extract temporal features. We conducted comprehensive experiments using both our collected dataset and the open dataset RadHar. The experimental outcomes demonstrate that our framework not only improves inference accuracy but also maintains satisfactory real-time performance on embedded platforms with constrained computational resources. When compared with state-of-the-art (SOTA) methods, our framework significantly enhances inference speed while retaining competitive inference accuracy. Codes and dataset are available at https://github.com/Feiyuyu0503/FastHAR.

AI-Assisted Food Intake Activity Recognition Using 3D Mmwave Radars.

Memory-Efficient High-Accuracy Food Intake Activity Recognition with 3D Mmwave Radars

A Novel Real-Time Multi-Person Activity Recognition System Using Millimeter-Wave SISO Radar Data

A Novel Multiperson Activity Recognition Algorithm Based on Point Clouds Measured by Millimeter-Wave MIMO Radar

Noninvasive Human Activity Recognition Using Millimeter-Wave Radar

Eat-Radar: Continuous Fine-Grained Intake Gesture Detection Using FMCW Radar and 3D Temporal Convolutional Network with Attention

A Dataset of Food Intake Activities Using Sensors with Heterogeneous Privacy Sensitivity Levels.

AI-Powered Non-Contact In-Home Gait Monitoring and Activity Recognition System Based on mm-Wave FMCW Radar and Cloud Computing

Human Activity Recognition Based on Deep Learning and Micro-Doppler Radar Data

Fast Human Action Recognition Via Millimeter Wave Radar Point Cloud Sequences Learning

Mmbehavior: Human Activity Recognition System of Millimeter-Wave Radar Point Clouds Based on Deep Recurrent Neural Network

Radar-based human activity recognition with adaptive thresholding towards resource constrained platforms

Human Activity Recognition via Wi-Fi and Inertial Sensors With Machine Learning

A deep learning based wearable system for food and drink intake recognition

WiEat: Fine-grained Device-free Eating Monitoring Leveraging Wi-Fi Signals.

Patient activity recognition using radar sensors and machine learning

EatingTrak: Detecting Fine-grained Eating Moments in the Wild Using a Wrist-mounted IMU.

Indoor Human Activity Recognition Using Millimeter-Wave Radio Signals

RF Sensing for Continuous Monitoring of Human Activities for Home Consumer Applications

Unsupervised Doppler Radar-Based Activity Recognition for e-Healthcare

Millimeter Wave Radar-based Human Activity Recognition for Healthcare Monitoring Robot