Abstract:The relationships between muscle movements and neural signals make it possible to decode silent speech based on neuromuscular activities. The decoding can be formulated as a supervised classification task. The electromyography (EMG) captured from surface articulatory muscles contains useful information that can help assist in decoding of speech. Spectrograms obtained from EMG have a wealth of information relating to the decoding, but have not yet been fully explored. In addition, the decoding results are often uncertain. Therefore, it is important to quantify the prediction confidence. This paper aims to improve the decoding performance by representing time series signals as spectrograms and utilising Inductive Conformal Prediction (ICP) to provide predictions with confidence. All EMG data are recorded on six dedicated facial muscles while participants recite the displayed words subvocally. Three pre-trained convolutional models of MobileNet-V1, ResNet18 and Xception are used to extract features from spectrograms for classification. Both bidirectional Long-Short Time Memory (Bi-LSTM) and Gate Recurrent Unit (GRU) classifiers are used for prediction. Furthermore, an ICP decoder based on Bi-LSTM is built to provide guaranteed predictions for each example at a specified confidence level. The proposed method of combining feature extraction based on Xception and classification using Bi-LSTM gives a higher accuracy of 0.87 than other methods. ICP outputs confidence measurements for each example that can help users to evaluate the reliability of new predictions. Experimental results demonstrate the practical usefulness in decoding articulatory neuromuscular activity and the advantages in applying ICP.

Decoding silent speech from high-density surface electromyographic data using transformer

Decoding Silent Speech Based on High-Density Surface Electromyogram Using Spatiotemporal Neural Network

Silent Speech Recognition Based on Surface Electromyography

Silent Speech Decoding Using Spectrogram Features Based on Neuromuscular Activities

Encoder-Decoder Architectures for Silent Speech Recognition Based on High-density Surface Electromyogram

Speech neuromuscular decoding based on spectrogram images using conformal predictors with Bi-LSTM.

SVIT‐SSR: A sEMG‐based vision transformer approach for silent speech recognition

sEMG-based technology for silent voice recognition

Silent Speech Recognition based on sEMG and EEG Signals

Design and implementation of a silent speech recognition system based on sEMG signals: A neural network approach

Hybrid Silent Speech Interface Through Fusion of Electroencephalography and Electromyography

Attention Bidirectional LSTM Networks Based Mime Speech Recognition Using Semg Data

Silent Speech Recognition Based on Surface Electromyography Using a Few Electrode Sites under the Guidance from High-Density Electrode Arrays.

Quality-aware Aggregated Conformal Prediction for Silent Speech Recognition

Convolutional Neural Network applied in mime speech recognition using sEMG data

Silent Speech Recognition Based on High-Density Surface Electromyogram Using Hybrid Neural Networks

Extracting Spatial Muscle Activation Patterns in Facial and Neck Muscles for Silent Speech Recognition Using High-Density sEMG

Exploration on Channel-interactive Features in Silent Speech Recognition

Feature selection of mime speech recognition using surface electromyography data

Sequence-to-Sequence Voice Reconstruction for Silent Speech in a Tonal Language

A novel silent speech recognition approach based on parallel inception convolutional neural network and Mel frequency spectral coefficient