Real-time Arm Gesture Recognition in Smart Home Scenarios via MillimeterWave Sensing

Haipeng Liu,Yuheng Wang,Anfu Zhou,Hanyue He,Wei Wang,Kunpeng Wang,Peilin Pan,Yixuan Lu,Liang Liu,Huadong Ma
DOI: https://doi.org/10.1145/3432235
2020-01-01
Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
Abstract:"In air" gesture recognition using millimeter wave (mmWave) radar and its applications in natural human-computer-interaction for smart home has shown its potential. However, the state-of-the-art works still fall short in terms of limited gesture space, vulnerable to surrounding interference, and off-line recognition. In this paper, we propose mHomeGes, a real-time mmWave arm gesture recognition system for practical smart home-usage. To this end, we first distill arm gesture's position and dynamic variation, and then custom-design a lightweight convolution neural network to recognize fine-grained gestures. Next, we propose a user discovery method to focus on the target human gesture, thus eliminating the adverse impact of surrounding interference. Finally, we design a hidden Markov model-based voting mechanism to handle continuous gesture signals at run-time, which leads to continuous gesture recognition in real-time. We implement mHomeGes on a commodity mmWave radar and also perform a user study, which demonstrates that mHomeGes achieves high recognition accuracy above 95.30% in real-time across various smart home scenarios, regardless of the impact of surrounding movements, concurrent gestures, human physiological conditions, and outer packing materials. Moreover, we have also publicly archived a mmWave gesture data-set collected during developing mHomeGes, which consists of about 22,000 instances from 25 persons and may have an independent value of facilitating future research.
What problem does this paper attempt to address?