Abstract:In this paper, a new speech driven 3-D geometric tongue model is constructed. The constructed 3-D tongue shape is controlled with control points on 2-D midsagittal tongue curve, and speech-driven inverse estimation based on the constructed model is evaluated by empirical data. X-Ray 2-D vocal tract motion videos are tagged for the midsagittal tongue motion, and static 3-D vocal tracts of 20 phonemes are collected with MRI for the realistic 3-D tongue shape. MFCC are calculated from the videos as acoustic features, and are then used in a LSTM-RNN to predict the control points movement of the tongue shape. Three geometrically intuitive control points are selected to represent and calculate the midsagittal line of the tongue through linear regression. Cross-sections on the central lines of the tongues, whose height, width and angle are then predicted from the midsagittal line, are reconstructed with geometric curves, and the shape of each cross-section are then placed on the midsagittal line to get the overall predicted moving grid of the 3-D tongue. In this 3-D tongue model, acoustic features and realistic tongue motion are mapped directly to preserve more realistic articulatory details, and the control points are intuitive for non-experts to control the model, and the geometric tongue shapes predicted are comparable with realistic tongue dynamics. Based on the proposed method, the speech-driven prediction is evaluated with the realistic data, which proved this proposed method feasible.

A novel 3D geometric articulatory model

A Novel Method for Constructing 3d Geometric Articulatory Models

An articulatory model of standard Chinese using MRI and X-ray movie

MRI-based 3D Model of Spoken Lips

An Improved 3D Geometric Tongue Model

Speech production of vowel sequences using a physiological articulatory model

Visualization of Mandarin articulation by using a physiological articulatory model

A 3D Geometry Model of Vocal Tract Based on Smart Internet of Things

A Speech-Driven 3-D Tongue Model with Realistic Movement in Mandarin Chinese.

A Study of Mandarin Chinese Using X-Ray and MRI

Estimation of vocal tract shapes from speech sounds with a physiological articulatory model

Improvement of a Physiological Articulatory Model for Synthesis of Vowel Sequences

Acoustic Analysis of the Vocal Tract from a 3D Physiological Articulatory Model by Finite-Difference Time-Domain Method

A Physiological Model of the Tongue and Jaw for Simulating Deformation in the Midsagittal and Parasagittal Planes

Detailed Morphological Analysis of Mandarin Sustained Steady Vowels

Radius Vector-Driven 3-D Mandarin Vocal Tract Model

A new method of reconstructing the human laryngeal architecture using micro-MRI.

Improvements of a Physiological Articulatory Model in Construction and Control Strategy

Estimation Of Vocal Tract Area Function For Mandarin Vowel Sequences Using Mri

A Design of Laryngeal Structures for a Physiological Articulatory Model

Speech synthesis of VCV sequence using a physiological articulatory model