Abstract:In this paper, a new speech driven 3-D geometric tongue model is constructed. The constructed 3-D tongue shape is controlled with control points on 2-D midsagittal tongue curve, and speech-driven inverse estimation based on the constructed model is evaluated by empirical data. X-Ray 2-D vocal tract motion videos are tagged for the midsagittal tongue motion, and static 3-D vocal tracts of 20 phonemes are collected with MRI for the realistic 3-D tongue shape. MFCC are calculated from the videos as acoustic features, and are then used in a LSTM-RNN to predict the control points movement of the tongue shape. Three geometrically intuitive control points are selected to represent and calculate the midsagittal line of the tongue through linear regression. Cross-sections on the central lines of the tongues, whose height, width and angle are then predicted from the midsagittal line, are reconstructed with geometric curves, and the shape of each cross-section are then placed on the midsagittal line to get the overall predicted moving grid of the 3-D tongue. In this 3-D tongue model, acoustic features and realistic tongue motion are mapped directly to preserve more realistic articulatory details, and the control points are intuitive for non-experts to control the model, and the geometric tongue shapes predicted are comparable with realistic tongue dynamics. Based on the proposed method, the speech-driven prediction is evaluated with the realistic data, which proved this proposed method feasible.

A Nonlinear 3D Geometric Tongue Model

An Improved 3D Geometric Tongue Model

A Speech-Driven 3-D Tongue Model with Realistic Movement in Mandarin Chinese.

A Multilinear Tongue Model Derived from Speech Related MRI Data of the Human Vocal Tract

A novel 3D geometric articulatory model

3D Whole Tooth Model from CT Volume Using Thin-Plate Splines.

Tongue Shape Synthesis Based on Active Shape Model.

A Physiological Model of the Tongue and Jaw for Simulating Deformation in the Midsagittal and Parasagittal Planes

THE APPLICATION OF RAPID PROTOTYPING IN THE ORTHOGNATHIC SURGERY

The Modeling of Tongue Tip in Standard Chinese Using MRI

A Computational Tongue Model and its Clinical Application

A Novel Method for Constructing 3d Geometric Articulatory Models

On a Mathematical Midsagittal Contour Model of Tongue

Tongue Segmentation with Geometrically Constrained Snake Model

Geometrical Analysis of the Tongue Muscles Based on MRI and Functional Modeling of the Tongue

A Hybrid Approach to Extracting Tooth Models from CT Volumes

3D dynamic tongue visualization based on DirectX

Extraction of Tongue Contour in Real-Time Magnetic Resonance Imaging Sequences

Inverse Biomechanical Modeling of the Tongue via Machine Learning and Synthetic Training Data

3D Reconstruction of Tongue Based on Binocular Stereo

Tongue shape conversion with non-parallel training data