Abstract:Total laryngectomy (TL) is as a well-established treatment for advanced laryngeal malignancies, entailing the complete removal of the larynx. Speech rehabilitation following TL is crucial for improving the quality of life and facilitating social reintegration. Electrolaryngeal (EL) speech, a widely used voice restoration technique utilizing external excitation signals, often produces artificial and monotonous sound quality. Efforts to enhance EL speech include the application of statistical voice conversion and neural approaches to speech enhancement. These approaches typically aim to map spectral features into acoustic characteristics, including the fundamental frequency ( ). However, challenges arise owing to substantial discrepancies and pattern differences between extracted features for EL and normal speech, compounded by limited clinical training data. To address this issue, we explored pattern prediction based on frame-wise phoneme information using bidirectional long short-term memory recurrent neural networks. Beyond direct predictions based on phoneme labels, we expanded our analysis to include real-valued phoneme embeddings and conducted predictions for clustered embeddings representing low-dimensional input representations. Our findings demonstrate that both regression and classification predictive modeling can map frame-wise phoneme information into natural patterns. Additionally, phoneme labels can be considered as shared features between EL and normal speech, allowing for improved prediction accuracies by incorporating phoneme information from normal speech into the training sets for EL speech. Furthermore, by learning phoneme embeddings and creating input features for prediction based on the clustering of these embeddings, accurate patterns can be predicted, and the challenge of finding a strategy to reduce the dimensionality of the input features can be effectively alleviated.

Polyphone Disambiguation and Accent Prediction Using Pre-Trained Language Models in Japanese TTS Front-End

Investigation of Japanese PnG BERT language model in text-to-speech synthesis for pitch accent language

ViSPer: A Multilingual TTS Approach Based on VITS Using Deep Feature Loss

A Polyphone BERT for Polyphone Disambiguation in Mandarin Chinese

Dict-TTS: Learning to Pronounce with Prior Dictionary Knowledge for Text-to-Speech

Cross-Dialect Text-To-Speech in Pitch-Accent Language Incorporating Multi-Dialect Phoneme-Level BERT

Data Augmentation for Long-Tailed and Imbalanced Polyphone Disambiguation in Mandarin

Towards Transfer Learning for End-to-End Speech Synthesis from Deep Pre-Trained Language Models.

Japanese Neural Incremental Text-to-Speech Synthesis Framework With an Accent Phrase Input

External Knowledge Augmented Polyphone Disambiguation Using Large Language Model

Improving Prosody with Linguistic and Bert Derived Features in Multi-Speaker Based Mandarin Chinese Neural TTS

Improving Mispronunciation Detection with Wav2vec2-based Momentum Pseudo-Labeling for Accentedness and Intelligibility Assessment

Improving Prosody for Unseen Texts in Speech Synthesis by Utilizing Linguistic Information and Noisy Data

Pronunciation Erroneous Tendency Detection with Language Adversarial Represent Learning.

Advancing Audio Emotion and Intent Recognition with Large Pre-Trained Models and Bayesian Inference

An acoustic model adaptation using HMM-based speech synthesis

Accent Estimation of Japanese Words from Their Surfaces and Romanizations for Building Large Vocabulary Accent Dictionaries

Applying Syntax$\unicode{x2013}$Prosody Mapping Hypothesis and Prosodic Well-Formedness Constraints to Neural Sequence-to-Sequence Speech Synthesis

Phonetic Enhanced Language Modeling for Text-to-Speech Synthesis

Improving Speech Prosody of Audiobook Text-to-Speech Synthesis with Acoustic and Textual Contexts

An Investigation of Fundamental Frequency Pattern Prediction for Japanese Electrolaryngeal Speech Enhancement Based on Frame-Wise Phoneme Representations