Abstract:The relationships between muscle movements and neural signals make it possible to decode silent speech based on neuromuscular activities. The decoding can be formulated as a supervised classification task. The electromyography (EMG) captured from surface articulatory muscles contains useful information that can help assist in decoding of speech. Spectrograms obtained from EMG have a wealth of information relating to the decoding, but have not yet been fully explored. In addition, the decoding results are often uncertain. Therefore, it is important to quantify the prediction confidence. This paper aims to improve the decoding performance by representing time series signals as spectrograms and utilising Inductive Conformal Prediction (ICP) to provide predictions with confidence. All EMG data are recorded on six dedicated facial muscles while participants recite the displayed words subvocally. Three pre-trained convolutional models of MobileNet-V1, ResNet18 and Xception are used to extract features from spectrograms for classification. Both bidirectional Long-Short Time Memory (Bi-LSTM) and Gate Recurrent Unit (GRU) classifiers are used for prediction. Furthermore, an ICP decoder based on Bi-LSTM is built to provide guaranteed predictions for each example at a specified confidence level. The proposed method of combining feature extraction based on Xception and classification using Bi-LSTM gives a higher accuracy of 0.87 than other methods. ICP outputs confidence measurements for each example that can help users to evaluate the reliability of new predictions. Experimental results demonstrate the practical usefulness in decoding articulatory neuromuscular activity and the advantages in applying ICP.

Voiced/Unvoiced Classification Recovery In The Speech Decoder Based On Gmm

Voiced/unvoiced Classification Recovery in GMM Speech Decoders

Voiced/Unvoiced Parameters Recovery Based on Second-Order Hidden Markov Model

Applying Support Vector Machines to Voice Activity Detection

Improved Recovery Algorithm for Unvoiced/voiced Parameters Based on GMM

Speech neuromuscular decoding based on spectrogram images using conformal predictors with Bi-LSTM.

HMM estimation of energy contours in speech decoders

Realtime robust speech communication based on iterative joint source-channel decoding and demodulation algorithm for MELP vocoder

Voice Activity Detection Using Wavelets Multiresolution Spectrum and Short-time Adaptive Audio Mixing Algorithm

Improved Memoryless GMM VQ for Speech Line Spectral Frequencies

Voice Activity Detection Based on Wavelet Multiresolution Spectrum

Voiced/unvoiced Decision Algorithm for HMM-based Speech Synthesis

Improvement of Voiced-Unvoiced Classification in Vocoders

Voice Conversion Based on Gaussian Mixture Modules with Minimum Distance Spectral Mapping

Fuzzy Unvoiced/voiced Decision-Making for Sub-Bands for SELP Vocoder

Voice Activity Detection Based on LPCC and Spectrum Entropy

A hybrid GMM and codebook mapping method for spectral conversion

GMM-based Voice Conversion with Explicit Modelling on Feature Transform

A hybrid method to convert acoustic features for voice conversion

A Fractional Bit Allocation Algorithm Based on Mixed Excitation Linear Prediction

A Vocal Password System Based on Multi-Dimension Feature Classifier in Score Domain