M-Gesture : Person-Independent Real-Time In-Air Gesture Recognition Using Commodity Millimeter Wave Radar
Haipeng Liu,Anfu Zhou,Zihe Dong,Yuyang Sun,Jiahe Zhang,Liang Liu,Huadong Ma,Jianhua Liu,Ning Yang
DOI: https://doi.org/10.1109/jiot.2021.3098338
IF: 10.6
2022-03-01
IEEE Internet of Things Journal
Abstract:Millimeter wave (mmWave) sensing promises to enable contactless and high-precision "in-air" gesture-based human–computer interaction (HCI). While previous works have demonstrated its feasibility, they require tedious gesture collecting for person-independent recognition and they operate in an off-line mode without considering practical issues, such as segmenting gesture and recognition latency. In this work, we propose M-Gesture, a person-independent real-time mmWave gesture recognition solution. We first build a compact gesture model with a custom-designed neural network to distill the unique features underlying each gesture, while suppressing personalized discrepancy across different users without extra collection and retraining. Furthermore, we design a system status transition (SST) to decide when a gesture begins and ends, which enables automatic gesture segmentation and hence real-time recognition. We prototype M-Gesture on a commodity mmWave sensor and demonstrate its advantages using two practical applications: 1) a contactless music player and 2) camera. Extensive experiments and user studies show that M-Gesture has an accuracy of 99% and a short response latency within 25 ms. Moreover, we also collect and release a comprehensive mmWave gesture data set consisting of 54 620 instances from 144 persons, which may have an independent value of facilitating future research.
computer science, information systems,telecommunications,engineering, electrical & electronic