Abstract:Extracting speech information from vibration response signals is a typical system identification problem, and the traditional method is too sensitive to deviations such as model parameters, noise, boundary conditions, and position. A method was proposed to obtain speech signals by collecting vibration signals of vibroacoustic systems for deep learning training in the work. The vibroacoustic coupling finite element model was first established with the voice signal as the excitation source. The vibration acceleration signals of the vibration response point were used as the training set to extract its spectral characteristics. Training was performed by two types of networks: fully connected, and convolutional. And it is found that the Fully Connected network prediction model has faster Rate of convergence and better quality of extracted speech. The amplitude spectra of the output speech signals (network output) and the phase of the vibration signals were used to convert extracted speech signals back to the time domain during the test set. The simulation results showed that the positions of the vibration response points had little effect on the quality of speech recognition, and good speech extraction quality can be obtained. The noises of the speech signals posed a greater influence on the speech extraction quality than the noises of the vibration signals. Extracted speech quality was poor when both had large noises. This method was robust to the position deviation of vibration responses during training and testing. The smaller the structural flexibility, the better the speech extraction quality. The quality of speech extraction was reduced in a trained system as the mass of node increased in the test set, but with negligible differences. Changes in boundary conditions did not significantly affect extracted speech quality. The speech extraction model proposed in the work has good robustness to position deviations, quality deviations, and boundary conditions.

Vocal Cord Vibration Signal Recognition Model Based on Feature Engineering Preprocessing

Speech neuromuscular decoding based on spectrogram images using conformal predictors with Bi-LSTM.

Decoding Throat-language Using Flexibility Sensors with Machine Learning

Speech extraction from vibration signals based on deep learning

Deep‐Learning‐Enabled MXene‐Based Artificial Throat: Toward Sound Detection and Speech Recognition

Leveraging Laryngograph Data for Robust Voicing Detection in Speech

Silent Speech Recognition Based on Surface Electromyography

Mixed-modality speech recognition and interaction using a wearable artificial throat

Smart Dysphagia Detection System with Adaptive Boosting Analysis of Throat Signals.

Robust Vocal Quality Feature Embeddings for Dysphonic Voice Detection

Bioinspired dual-channel speech recognition using graphene-based electromyographic and mechanical sensors

Research On Singing Voice Detection Based On A Long-Term Recurrent Convolutional Network With Vocal Separation And Temporal Smoothing

Optimal Deep Learning-Based Vocal Fold Disorder Detection and Classification Model on High-Speed Video Endoscopy

Electromyogram-strain synergetic intelligent artificial throat

Speaking without vocal folds using a machine-learning-assisted wearable sensing-actuation system

Dysphagia diagnosis system with integrated speech analysis from throat vibration

Multi-Band Speech Tensor Decomposition for Interactive Feature Extraction in Early Dysphagia Screening.

Exploring Large Scale Pre-Trained Models for Robust Machine Anomalous Sound Detection

A VIBRATION RECOGNITION METHOD BASED ON DEEP LEARNING AND SIGNAL PROCESSING

Automatic Detection System for Velopharyngeal Insufficiency Based on Acoustic Signals from Nasal and Oral Channels

DCIM-AVSR : Efficient Audio-Visual Speech Recognition via Dual Conformer Interaction Module