Abstract:Recently, several deep-learning (DL) techniques using different types of 2-D representations of radar returns, have been developed for radar-based human activity recognition (HAR). Most of these DL techniques involve a fusion approach (either at the feature level (intermediate) or at the decision level (late)) as the information obtained from one 2-D radar representation supplements the information obtained from another 2-D radar representation for enhanced HAR. The inputs to these fusion-based DL techniques are red, green, and blue (RGB) images of 2-D representations of radar returns. The information contained in the 2-D representations is completely mapped to color (RGB) domains. However, none of the DL techniques exploit this color information explicitly for HAR. This work proposes a novel lightweight multidomain multilevel fused patch-based learning model that exploits individual color domain information of RGB images of 2-D representations, namely, range-time, range-Doppler, and spectrograms for enhanced HAR using radars. This work proposes a novel domain-level (early) fusion of 2-D representations of radar returns based on the color domain information. Individual color planes (R, G, B) of 2-D representations are fused together to form consolidated three-channel images that serve as input to an isotropic patch-based learning model called convolutional mixer (convMixer). The early (domain) fused three-channel images are used as inputs to an attentional feature level (intermediate) fusion-based convMixer models. The performance of the proposed model is evaluated using a publicly available radar signatures dataset of human activities. The proposed model outperforms the state-of-the-art significantly using a location-wise testing strategy, which eliminates the possibility of data leakage.

FMCW Radar Sensor Based Human Activity Recognition using Deep Learning

Multi-frequency and multi-domain human activity recognition based on SFCW radar using deep learning

Human Activity Recognition Method Based on FMCW Radar Sensor with Multi-Domain Feature Attention Fusion Network

Human Activity Recognition Based on Deep Learning and Micro-Doppler Radar Data

Radar Human Activity Recognition with an Attention-Based Deep Learning Network

Human Activity Recognition Based on Frequency-Modulated Continuous Wave and DenseNet

Human Activity Recognition Based on Mixed CNN With Radar Multi-Spectrogram

FMCW Radar Human Action Recognition Based on Asymmetric Convolutional Residual Blocks

Parallel LSTM-CNN Network with Radar Multispectrogram for Human Activity Recognition

Deep Learning Techniques for Radar-Based Continuous Human Activity Recognition

Noninvasive Human Activity Recognition Using Millimeter-Wave Radar

Contactless Human Activity Recognition using Deep Learning with Flexible and Scalable Software Define Radio

Continuous Human Activity Recognition With Distributed Radar Sensor Networks and CNN–RNN Architectures

Angle-Insensitive Human Motion and Posture Recognition Based on 4D Imaging Radar and Deep Learning Classifiers

A Parameter Estimation and Deep Learning Hybrid Extraction Network for Multi-Directional Human Activity Recognition Based on Mmwave Radar

Human Activity Recognition Based on Multidomain Fusion Network for LFMCW Radar

Radar Based Activity Recognition using CNN-LSTM Network Architecture

Sparsity-based Human Activity Recognition with PointNet using a Portable FMCW Radar

RF-Based Human Activity Recognition Using Signal Adapted Convolutional Neural Network

CAE-MAS: Convolutional Autoencoder Interference Cancellation for Multiperson Activity Sensing With FMCW Microwave Radar

Radar-Based Human Activity Recognition Using Multidomain Multilevel Fused Patch-Based Learning