Abstract:Radar-based gesture recognition combined machine learning methods can achieve excellent performance and has been utilized in a wide variety of applications. However, large-scale radar signal processing suffers from the risk of privacy leakage and the problem of limited computing resources. In this paper, we propose a federated learning framework based on the spiking neural network to solve above limitations in radar gesture recognition. First, we build a distributed federated learning system with good privacy protection to process radar data collaboratively, which only exchanges the model parameters between clients to train the model, without the need for transferring the data itself. Second, we train the spiking neural network using the spike timing dependent plasticity with biological interpretability, which is more compatible with the biological characteristics and can significantly reduce the energy consumption. Our creative combination of low-power spiking neural network and federated learning thus address the issue of resource constraint of edge nodes in federated learning. Furthermore, we introduce the weight pruning technique to the federated learning training process to reduce the model's communication cost. We conduct experiments on the 8GHz radar gesture dataset and Google Soli dataset to test the proposed model's validity. We evaluate the model's performance across various participating devices and examine the spiking neural network's capacity to recognize gestures under independent identical distribution and non-independent identical distribution strategies in federated learning. We also analyze the energy consumption and communication cost of the proposed model, demonstrating the spike timing dependent plasticity-based algorithm is more energy efficient than the traditional backpropagation as well as the backpropagation through time algorithms.

Federated Learning for Radar Gesture Recognition Based on Spike Timing Dependent Plasticity

A RD-T Network for Hand Gesture Recognition Based on Millimeter-Wave Sensor

Human Recognition with the Optoelectronic Reservoir-Computing-based Micro-Doppler Radar Signal Processing.

Simple and Efficient Gesture Recognition Based on Frequency-Modulated Continuous Wave Radar

RT-SCNNs: real-time spiking convolutional neural networks for a novel hand gesture recognition using time-domain mm-wave radar data

Gesture Recognition System Using 24 GHz FMCW Radar Sensor Realized on Real-Time Edge Computing Platform

Sparsity Aware Dynamic Gesture Recognition Using Radar Sensors with Angular Diversity

A Gesture Recognition Method Based On Spiking Neural Networks For Cognition Development

Hand Gesture Recognition Using IR-UWB Radar with Spiking Neural Networks

TinyRadarNN: Combining Spatial and Temporal Convolutional Neural Networks for Embedded Gesture Recognition with Short Range Radars

A Robust Hand Gesture Sensing and Recognition Based on Dual-Flow Fusion with FMCW Radar.

Sparsity Aware Dynamic Gesture Classification Using Dual-Band Radar

Cross-modal Learning of Graph Representations using Radar Point Cloud for Long-Range Gesture Recognition

Resonate-and-Fire Spiking Neurons for Target Detection and Hand Gesture Recognition: A Hybrid Approach

Radar Based Gesture Recognition in Embedded Systems

A Lightweight Hand-Gesture Recognition Network With Feature Fusion Prefiltering and FMCW Radar Spatial Angle Estimation

RadarNet: Efficient Gesture Recognition Technique Utilizing a Miniature Radar Sensor

Efficient Gesture Recognition on Spiking Convolutional Networks Through Sensor Fusion of Event-Based and Depth Data

Millimeter wave gesture recognition using multi-feature fusion models in complex scenes

Sign Language/Gesture Recognition on OOD Target Domains Using UWB Radar

Gesture Recognition in Millimeter-Wave Radar Based on Spatio-Temporal Feature Sequences