GesturePrint: Enabling User Identification for mmWave-based Gesture Recognition Systems

Lilin Xu,Keyi Wang,Chaojie Gu,Xiuzhen Guo,Shibo He,Jiming Chen

DOI: https://doi.org/10.48550/arXiv.2408.05358

2024-07-26

Abstract:The millimeter-wave (mmWave) radar has been exploited for gesture recognition. However, existing mmWave-based gesture recognition methods cannot identify different users, which is important for ubiquitous gesture interaction in many applications. In this paper, we propose GesturePrint, which is the first to achieve gesture recognition and gesture-based user identification using a commodity mmWave radar sensor. GesturePrint features an effective pipeline that enables the gesture recognition system to identify users at a minor additional cost. By introducing an efficient signal preprocessing stage and a network architecture GesIDNet, which employs an attention-based multilevel feature fusion mechanism, GesturePrint effectively extracts unique gesture features for gesture recognition and personalized motion pattern features for user identification. We implement GesturePrint and collect data from 17 participants performing 15 gestures in a meeting room and an office, respectively. GesturePrint achieves a gesture recognition accuracy (GRA) of 98.87% with a user identification accuracy (UIA) of 99.78% in the meeting room, and 98.22% GRA with 99.26% UIA in the office. Extensive experiments on three public datasets and a new gesture dataset show GesturePrint's superior performance in enabling effective user identification for gesture recognition systems.

Signal Processing,Computer Vision and Pattern Recognition,Human-Computer Interaction,Machine Learning

What problem does this paper attempt to address?

### What problems does this paper attempt to solve?

This paper aims to solve the problem that millimeter - wave (mmWave) radar gesture recognition systems are unable to distinguish different users. Although the existing millimeter - wave - based gesture recognition methods can accurately recognize predefined gestures, they cannot identify the specific users who perform these gestures. This is an important limitation for many application scenarios that require personalized interaction.

#### Specific problem description:

1. **Lack of user identification**: Current millimeter - wave gesture recognition systems cannot identify the user identities of those who perform gestures, which limits the functions and user experiences of the systems in practical applications.
2. **Requirement for personalized interaction**: In application scenarios such as smart homes and music players, identifying user identities can significantly enhance the user experience. For example, adjusting the music list according to users' preferences or personalizing gesture operations.
3. **Limitations of existing methods**: Although there are some gesture recognition and user identification solutions based on other modalities (such as video, IMU, WiFi), they cannot be directly applied to millimeter - wave radar data because of the differences in data formats and feature representations of different modalities.

#### Solution:

To solve the above problems, the authors propose a system named **GesturePrint**. GesturePrint is the first millimeter - wave radar system that can simultaneously achieve gesture recognition and user identification. It solves the above problems in the following ways:

- **Effective gesture capture**: Adopt the parameter - adaptive sliding - window method to reliably segment gestures and remove irrelevant noise points, ensuring the accuracy and integrity of gesture data.
- **Efficient feature extraction**: Introduce a specially designed network architecture **GesIDNet**, which combines an attention mechanism and a multi - level feature - fusion module and can extract unique gesture features and personalized motion - pattern features from sparse point - cloud data.
- **Robust feature learning**: Through data augmentation and a carefully designed network structure, the system can remain robust in the face of differences (such as speed, distance, etc.) when the same user performs the same gesture multiple times.

#### Experimental verification:

The authors verified the performance of GesturePrint by collecting data from 17 participants performing 15 gestures in meeting - room and office environments. The experimental results show that GesturePrint achieved a 98.87% gesture - recognition accuracy and a 99.78% user - identification accuracy in the meeting - room environment, and also reached a similar high accuracy in the office environment. In addition, the authors also conducted extensive experiments on three public datasets and a newly constructed ASL - gesture dataset, further demonstrating the superior performance of GesturePrint.

In conclusion, GesturePrint provides a low - cost and efficient solution, enabling millimeter - wave - based gesture - recognition systems to have user - identification capabilities, thereby enhancing the personalization and intelligence levels of gesture interactions.