Abstract:Vocal fold (VF) vibrations are the primary source of human phonation. High-speed video (HSV) endoscopy enables the computation of descriptive VF parameters for assessment of physiological properties of laryngeal dynamics, i.e., the vibration of the VFs. However, underlying biomechanical factors responsible for physiological and disordered VF vibrations cannot be accessed. In contrast, physically based numerical VF models reveal insights into the organ's oscillations, which remain inaccessible through endoscopy. To estimate biomechanical properties, previous research has fitted subglottal pressure-driven mass–spring–damper systems, as inverse problem to the HSV-recorded VF trajectories, by global optimization of the numerical model. A neural network trained on the numerical model may be used as a substitute for computationally expensive optimization, yielding a fast evaluating surrogate of the biomechanical inverse problem. This paper proposes a convolutional recurrent neural network (CRNN)-based architecture trained on regression of a physiological-based biomechanical six-mass model (6 MM). To compare with previous research, the underlying biomechanical factor "subglottal pressure" prediction was tested against 288 HSV ex vivo porcine recordings. The contributions of this work are two-fold: first, the presented CRNN with the 6 MM handles multiple trajectories along the VFs, which allows for investigations on local changes in VF characteristics. Second, the network was trained to reproduce further important biomechanical model parameters like VF mass and stiffness on synthetic data. Unlike in a previous work, the network in this study is therefore an entire surrogate of the inverse problem, which allowed for explicit computation of the fitted model using our approach. The presented approach achieves a best-case mean absolute error (MAE) of 133 Pa (13.9%) in subglottal pressure prediction with 76.6% correlation on experimental data and a re-estimated fundamental frequency MAE of 15.9 Hz (9.9%). In-detail training analysis revealed subglottal pressure as the most learnable parameter. With the physiological-based model design and advances in fast parameter prediction, this work is a next step in biomechanical VF model fitting and the estimation of laryngeal kinematics.

Estimating vocal tract geometry from acoustic impedance using deep neural network

Neural network-based estimation of biomechanical vocal fold parameters

Towards Automatic Speech Identification from Vocal Tract Shape Dynamics in Real-time MRI

Automatic segmentation of vocal tract articulators in real-time magnetic resonance imaging

A 3D Geometry Model of Vocal Tract Based on Smart Internet of Things

Vocal Tract Area Estimation by Gradient Descent

A Deep Neural Network Based Glottal Flow Model for Predicting Fluid-Structure Interactions during Voice Production

A comparative study of two-dimensional vocal tract acoustic modeling based on Finite-Difference Time-Domain methods

A Hybrid Method for Acoustic Analysis of the Vocal Tract During Vowel Production.

An extended two-dimensional vocal tract model for fast acoustic simulation of single-axis symmetric three-dimensional tubes

Uncertainty Quantification of Vibroacoustics with Deep Neural Networks and Catmull–Clark Subdivision Surfaces

Acoustic Impedance Inversion from Seismic Imaging Profiles Using Self Attention U-Net

3D Room Geometry Inference from Multichannel Room Impulse Response using Deep Neural Network

Neural Ambisonics encoding for compact irregular microphone arrays

Estimation Of Vocal Tract Area Function For Mandarin Vowel Sequences Using Mri

A deep recurrent approach for acoustic-to-articulatory inversion

Physics-Informed Neural Network for Volumetric Sound field Reconstruction of Speech Signals

Reconstructing the Dynamic Directivity of Unconstrained Speech

Predicting reflection patterns from binaural activity maps using deep neural networks

A new model for acoustic wave propagation and scattering in the vocal tract

Speaker dependent articulatory-to-acoustic mapping using real-time MRI of the vocal tract