MMG-HCI - A Non-contact Non-intrusive Real-Time Intelligent Human-Computer Interaction System.

Peixian Gong,Chunyu Wang,Lihua Zhang

DOI: https://doi.org/10.1007/978-3-030-93046-2_14

2021-01-01

Abstract:With the continuous development of science and technology, human-computer interaction (HCI) technology has been paid more and more attention by researchers. Traditional HCI methods, such as torch-screen, speech recognition and so on, have the problem of inconvenient application in some scenarios. The research of HCI technology based on millimeter-wave radar makes up for the deficiency of current HCI, and plays a very important role in its development. The main tasks of this paper are as follows: 1, Design of non-contact and non-invasive real-time human activity recognition system MMG-HCI. By analyzing the working features of millimeter-wave radar and combining them with the principle of computer multi-process synchronization, we designed a real-time system which integrates millimeter-wave radar data acquisition, point cloud data visualization, deep learning model inference and HCI. 2, Construction of millimeter-wave radar gesture dataset. Since there is no suitable open-source dataset at present, we collect and build our own dataset through the real-time system. 3, Training graph neural network (GNN) for HAR based on millimeter-wave radar. In order to classify the mmWave radar point cloud more accurately, we use the best performance MMPointGNN as the deep learning inference model of a real-time system. Experiments and demonstration show that we have designed an efficient, accurate, non-contact and non-invasive real-time HCI system.