TRANS-CNN-Based Gesture Recognition for mmWave Radar

Huafeng Zhang,Kang Liu,Yuanhui Zhang,Jihong Lin

DOI: https://doi.org/10.3390/s24061800

IF: 3.9

2024-03-12

Sensors

Abstract:In order to improve the real-time performance of gesture recognition by a micro-Doppler map of mmWave radar, the point cloud based gesture recognition for mmWave radar is proposed in this paper. Two steps are carried out for mmWave radar-based gesture recognition. The first step is to estimate the point cloud of the gestures by 3D-FFT and the peak grouping. The second step is to train the TRANS-CNN model by combining the multi-head self-attention and the 1D-convolutional network so as to extract the features in the point cloud data at a deeper level to categorize the gestures. In the experiments, TI mmWave radar sensor IWR1642 is used as a benchmark to evaluate the feasibility of the proposed approach. The results show that the accuracy of the gesture recognition reaches 98.5%. In order to prove the effectiveness of our approach, a simply 2Tx2Rx radar sensor is developed in our lab, and the accuracy of recognition reaches 97.1%. The results show that our proposed gesture recognition approach achieves the best performance in real time with limited training data in comparison with the existing methods.

engineering, electrical & electronic,chemistry, analytical,instruments & instrumentation

What problem does this paper attempt to address?

### Problems the Paper Aims to Solve

This paper aims to improve the real-time performance of gesture recognition using millimeter-wave radar (mmWave radar). Specifically, the paper proposes a point cloud-based gesture recognition method and designs a new model named TRANS-CNN by combining multi-head self-attention mechanism and 1D-convolutional network. This method aims to extract deeper features from point cloud data to achieve efficient and accurate gesture classification.

### Main Features of the Solution
1. **Point Cloud Data Processing**:
- Proposes a method to estimate gesture point clouds through 3D Fourier Transform (3D-FFT) and peak grouping.
- Uses point cloud data as input to address the issue of excessive redundant information present in micro-Doppler images.

2. **Model Architecture**:
- Combines multi-head self-attention mechanism with 1D-convolutional network to extract global and local information from point cloud data.
- This combination significantly reduces the overall complexity of the model, lowers hardware resource utilization, increases gesture computation speed, and achieves real-time gesture recognition.

3. **Experimental Validation**:
- Uses TI's mmWave radar sensor IWR1642 as a benchmark for experimental evaluation.
- Experimental results show that the method achieved a recognition accuracy of 98.5% with limited training data and can complete gesture recognition tasks within 1 second.

In summary, the paper addresses the issues of data redundancy and lack of real-time performance in existing methods through innovative point cloud data processing and an efficient TRANS-CNN model, enhancing the accuracy and real-time performance of gesture recognition.