Typing Everywhere with an EMG Keyboard: A Novel Myo Armband-Based HCI Tool

Zongkai Fu,Huiyong Li,Zhenchao Ouyang,Xuefeng Liu,Jianwei Niu

DOI: https://doi.org/10.1007/978-3-030-60245-1_17

2020-01-01

Abstract:To enhance users’ experience of inputting characters on mobile devices with small screens, this paper designed a novel virtual keyboard used on mobile devices. In particular, we introduce a novel virtual keyboard based on the MYO armband which is able to capture the electromyogram (EMG) signals of users when typing on any surfaces (such as human body or normal desktop). The actions of the three fingers are mapped to the nine keys of the T9 keyboard. After that, the signals of finger motions are translated into key sequences of the T9 keyboard. However, the identification of continuous finger motions is a critical challenge. To address the challenge, we convert the EMG signals in time domain into a 3D time-frequency map (each channel corresponds to the EMG unit of a frequency-domain feature), and extract the convolutional features with a 4-layer CNN (Convolutional Neural Network) module, an im2col module of Optical Character Recognition (OCR) and a Long Short-Term Memory (LSTM) module, and the final result is achieved as a probability graph of finger gestures. The Connection Temporal Classification (CTC) algorithm is adopted to find the best gesture sequence from the probability map. Experimental results show that our method can effectively identify different key sequences at three different input speeds with an average accuracy of 85.9%, and the integration testing with different volunteers shows that our method can achieve an average typing speed of 15.7 Word-Per-Minute (WPM).