Abstract:Recently, several deep-learning (DL) techniques using different types of 2-D representations of radar returns, have been developed for radar-based human activity recognition (HAR). Most of these DL techniques involve a fusion approach (either at the feature level (intermediate) or at the decision level (late)) as the information obtained from one 2-D radar representation supplements the information obtained from another 2-D radar representation for enhanced HAR. The inputs to these fusion-based DL techniques are red, green, and blue (RGB) images of 2-D representations of radar returns. The information contained in the 2-D representations is completely mapped to color (RGB) domains. However, none of the DL techniques exploit this color information explicitly for HAR. This work proposes a novel lightweight multidomain multilevel fused patch-based learning model that exploits individual color domain information of RGB images of 2-D representations, namely, range-time, range-Doppler, and spectrograms for enhanced HAR using radars. This work proposes a novel domain-level (early) fusion of 2-D representations of radar returns based on the color domain information. Individual color planes (R, G, B) of 2-D representations are fused together to form consolidated three-channel images that serve as input to an isotropic patch-based learning model called convolutional mixer (convMixer). The early (domain) fused three-channel images are used as inputs to an attentional feature level (intermediate) fusion-based convMixer models. The performance of the proposed model is evaluated using a publicly available radar signatures dataset of human activities. The proposed model outperforms the state-of-the-art significantly using a location-wise testing strategy, which eliminates the possibility of data leakage.

View-spatial-attention Augmentation on Multi-view Micro-Doppler Spectra for Hidden Human Activity Recognition

Sample Intercorrelation-Based Multidomain Fusion Network for Aquatic Human Activity Recognition Using Millimeter-Wave Radar.

A Novel Real-Time Multi-Person Activity Recognition System Using Millimeter-Wave SISO Radar Data

Indoor Activity and Vital Sign Monitoring for Moving People with Multiple Radar Data Fusion

Noninvasive Human Activity Recognition Using Millimeter-Wave Radar

Multiscale Residual Attention Network for Distinguishing Stationary Humans and Common Animals under Through-Wall Condition Using Ultra-Wideband Radar.

Application of multi-angle millimeter-wave radar detection in human motion behavior and micro-action recognition

Human Activity Recognition Based on Mixed CNN With Radar Multi-Spectrogram

Millimeter-Wave Radar Monitoring for Elder's Fall Based on Multi-View Parameter Fusion Estimation and Recognition

Direction-Independent Human Behavior Recognition Using Distributed Radar Sensor System and Hybrid Neural Network with Dual-View Attention

Human Activity Recognition Based on Multidomain Fusion Network for LFMCW Radar

A Multi-Domain Fusion Human Motion Recognition Method Based on Lightweight Network

Radar-Based Human Activity Recognition Using Multidomain Multilevel Fused Patch-Based Learning

Multi-frequency and multi-domain human activity recognition based on SFCW radar using deep learning

Innovative Dual-Decoupling CNN with Layer-wise Temporal-Spatial Attention for Sensor-Based Human Activity Recognition

Human Activity Recognition Method Based on FMCW Radar Sensor with Multi-Domain Feature Attention Fusion Network

Fusion Classification of Finer-grained Human Activity Using MIMO UWB Bio-Radar

Robust Human Detection under Visual Degradation via Thermal and mmWave Radar Fusion

Multi-Sensor Data Fusion and CNN-LSTM Model for Human Activity Recognition System

GCN-Enhanced Multidomain Fusion Network for Through-Wall Human Activity Recognition

Parallel LSTM-CNN Network with Radar Multispectrogram for Human Activity Recognition