Abstract:Since the advent of modern computing, researchers have striven to make the human–computer interface (HCI) as seamless as possible. Progress has been made on various fronts, e.g., the desktop metaphor (interface design) and natural language processing (input). One area receiving attention recently is voice activation and its corollary, computer-generated speech. Despite decades of research and development, most computer-generated voices remain easily identifiable as non-human. Prosody in speech has two primary components—intonation and rhythm—both often lacking in computer-generated voices. This research aims to enhance computer-generated text-to-speech algorithms by incorporating melodic and prosodic elements of human speech. This study explores a novel approach to add prosody by using machine learning, specifically an LSTM neural network, to add paralinguistic elements to a recorded or generated voice. The aim is to increase the realism of computer-generated text-to-speech algorithms, to enhance electronic reading applications, and improved artificial voices for those in need of artificial assistance to speak. A computer that is able to also convey meaning with a spoken audible announcement will also improve human-to-computer interactions. Applications for the use of such an algorithm may include improving high-definition audio codecs for telephony, renewing old recordings, and lowering barriers to the utilization of computing. This research deployed a prototype modular platform for digital speech improvement by analyzing and generalizing algorithms into a modular system through laboratory experiments to optimize combinations and performance in edge cases. The results were encouraging, with the LSTM-based encoder able to produce realistic speech. Further work will involve optimizing the algorithm and comparing its performance against other approaches.

STRAIGHTMORPH: A Voice Morphing Tool for Research in Voice Communication Sciences

Interactive tools for making temporally variable, multiple-attributes, and multiple-instances morphing accessible: Flexible manipulation of divergent speech instances for explorational research and education

A Chinese Voice Morphing Method Based on STRAIGHT

Voice Morphing: Two Identities in One Voice

MorphFader: Enabling Fine-grained Controllable Morphing with Text-to-Audio Models

Analysis of a Modern Voice Morphing Approach using Gaussian Mixture Models for Laryngectomees

SoundMorpher: Perceptually-Uniform Sound Morphing with Diffusion Model

OpenVoice: Versatile Instant Voice Cloning

Controlled AutoEncoders to Generate Faces from Voices

Soundgen: An open-source tool for synthesizing nonverbal vocalizations

FaceFormer: Speech-Driven 3D Facial Animation with Transformers

Audio is all in one: speech-driven gesture synthetics using WavLM pre-trained model

RAVE for Speech: Efficient Voice Conversion at High Sampling Rates

An improved method for voice conversion based on Gaussian mixture model

Make-A-Voice: Unified Voice Synthesis With Discrete Representation

Voice Synthesis Improvement by Machine Learning of Natural Prosody

VoiceLoop: Voice Fitting and Synthesis via a Phonological Loop

MASS: Multi-task anthropomorphic speech synthesis framework

Creating New Voices using Normalizing Flows

Giving Robots a Voice: Human-in-the-Loop Voice Creation and open-ended Labeling

An Overview of Voice Conversion and Its Challenges: From Statistical Modeling to Deep Learning