Abstract:Purpose: Current electromagnetic tongue tracking devices are not amenable for daily use and thus not suitable for silent speech interface and other applications. We have recently developed MagTrack, a novel wearable electromagnetic articulograph tongue tracking device. This study aimed to validate MagTrack for potential silent speech interface applications. Method: We conducted two experiments: (a) classification of eight isolated vowels in consonant-vowel-consonant form and (b) continuous silent speech recognition. In these experiments, we used data from healthy adult speakers collected with MagTrack. The performance of vowel classification was measured by accuracies. The continuous silent speech recognition was measured by phoneme error rates. The performance was then compared with results using data collected with commercial electromagnetic articulograph in a prior study. Results: The isolated vowel classification using MagTrack achieved an average accuracy of 89.74% when leveraging all MagTrack signals (x, y, z coordinates; orientation; and magnetic signals), which outperformed the accuracy using commercial electromagnetic articulograph data (only y, z coordinates) in our previous study. The continuous speech recognition from two subjects using MagTrack achieved phoneme error rates of 73.92% and 66.73%, respectively. The commercial electromagnetic articulograph achieved 64.53% from the same subject (66.73% using MagTrack data). Conclusions: MagTrack showed comparable results with the commercial electromagnetic articulograph when using the same localized information. Adding raw magnetic signals would improve the performance of MagTrack. Our preliminary testing demonstrated the potential for silent speech interface as a lightweight wearable device. This work also lays the foundation to support MagTrack's potential for other applications including visual feedback-based speech therapy and second language learning.

Cross-Speaker Silent-Speech Command Word Recognition Using Electro-Optical Stomatography

Silent Speech Decoding Using Spectrogram Features Based on Neuromuscular Activities

Attention Bidirectional LSTM Networks Based Mime Speech Recognition Using Semg Data

Speech neuromuscular decoding based on spectrogram images using conformal predictors with Bi-LSTM.

Research on Speaker-Depended Isolated-Word Speech Recognition System

Silent Speech Recognition Based on Surface Electromyography

Silent speech command word recognition using stepped frequency continuous wave radar

Decoding Silent Speech Commands from Articulatory Movements Through Soft Magnetic Skin and Machine Learning

An Audio-Visual Speech Recognition Framework Based on Articulatory Features.

Speech Recognition and a Cymatics Based Configurable Speech Perception

Speaker Recognition Technology Based on Lip Movement

Silenttalk: Lip Reading Through Ultrasonic Sensing on Mobile Phones

Exploiting Cross-domain And Cross-Lingual Ultrasound Tongue Imaging Features For Elderly And Dysarthric Speech Recognition

All-weather, natural silent speech recognition via machine-learning-assisted tattoo-like electronics

MagTrack: A Wearable Tongue Motion Tracking System for Silent Speech Interfaces

Speech Reconstruction from Silent Tongue and Lip Articulation By Pseudo Target Generation and Domain Adversarial Training

Silent Speech Eyewear Interface: Silent Speech Recognition Method Using Eyewear and an Ear-Mounted Microphone with Infrared Distance Sensors

Speech Reconstruction from Silent Lip and Tongue Articulation by Diffusion Models and Text-Guided Pseudo Target Generation

A Cross-Modal Approach to Silent Speech with LLM-Enhanced Recognition

Pronunciation recognition of English phonemes /\textipa{@}/, /æ/, /\textipa{A}:/ and /\textipa{2}/ using Formants and Mel Frequency Cepstral Coefficients

State-of-the-art in speaker recognition