Abstract:Recently, several deep-learning (DL) techniques using different types of 2-D representations of radar returns, have been developed for radar-based human activity recognition (HAR). Most of these DL techniques involve a fusion approach (either at the feature level (intermediate) or at the decision level (late)) as the information obtained from one 2-D radar representation supplements the information obtained from another 2-D radar representation for enhanced HAR. The inputs to these fusion-based DL techniques are red, green, and blue (RGB) images of 2-D representations of radar returns. The information contained in the 2-D representations is completely mapped to color (RGB) domains. However, none of the DL techniques exploit this color information explicitly for HAR. This work proposes a novel lightweight multidomain multilevel fused patch-based learning model that exploits individual color domain information of RGB images of 2-D representations, namely, range-time, range-Doppler, and spectrograms for enhanced HAR using radars. This work proposes a novel domain-level (early) fusion of 2-D representations of radar returns based on the color domain information. Individual color planes (R, G, B) of 2-D representations are fused together to form consolidated three-channel images that serve as input to an isotropic patch-based learning model called convolutional mixer (convMixer). The early (domain) fused three-channel images are used as inputs to an attentional feature level (intermediate) fusion-based convMixer models. The performance of the proposed model is evaluated using a publicly available radar signatures dataset of human activities. The proposed model outperforms the state-of-the-art significantly using a location-wise testing strategy, which eliminates the possibility of data leakage.

Few-shot Human Activity Recognition with Radar Micro-Doppler Spectrograms

Supervised Domain Adaptation for Few-Shot Radar-Based Human Activity Recognition

Semisupervised Human Activity Recognition with Radar Micro-Doppler Signatures

Radar Human Activity Recognition with an Attention-Based Deep Learning Network

A Survey of Deep Learning-Based Human Activity Recognition in Radar

Human Activity Recognition Based on Deep Learning and Micro-Doppler Radar Data

Unsupervised Domain Adaptation for Human Activity Recognition in Radar

Category-Extensible Human Activity Recognition Based on Doppler Radar by Few-Shot Learning

Multi-frequency and multi-domain human activity recognition based on SFCW radar using deep learning

Indoor Human Action Recognition Based on Millimeter-Wave Radar Micro-Doppler Signature

Radar-Based Human Activity Recognition Using Multidomain Multilevel Fused Patch-Based Learning

FMCW Radar Sensor Based Human Activity Recognition using Deep Learning

Human Activity Recognition Based on Mixed CNN With Radar Multi-Spectrogram

Human Activity Recognition Method Based on FMCW Radar Sensor with Multi-Domain Feature Attention Fusion Network

Parallel LSTM-CNN Network with Radar Multispectrogram for Human Activity Recognition

Angle-Insensitive Human Motion and Posture Recognition Based on 4D Imaging Radar and Deep Learning Classifiers

Person Identification with Limited Training Data Using Radar Micro‐doppler Signatures

Human Motion Recognition With Limited Radar Micro-Doppler Signatures

A Parameter Estimation and Deep Learning Hybrid Extraction Network for Multi-Directional Human Activity Recognition Based on Mmwave Radar

Resolution-Adaptive Micro-Doppler Spectrogram for Human Activity Recognition

Deep Learning Techniques for Radar-Based Continuous Human Activity Recognition