Abstract:As the development of deep learning, neural network (NN) based text-to-speech (TTS) that adopts deep neural networks as the model backbone for speech synthesis, has now become the mainstream technology for TTS. Compared to the previous TTS systems based on concatenative synthesis and statistical parametric synthesis, the NN based speech synthesis shows conspicuous advantages. It needs less requirement on human pre-processing and feature development, and brings high-quality voice in terms of both intelligibility and naturalness. However, robust NN based speech synthesis model typically requires a sizable set of high-quality data for training, which is expensive to collect especially in low-resource scenarios. It is worth investigating how to take advantage of low-quality material such as automatic speech recognition (ASR) data which can be easily obtained compared with high-quality TTS material. In this paper, we propose a pre-training technique framework to improve the performance of low-resource speech synthesis. The idea is to extend the training material of TTS model by using ASR based data augmentation method. Specifically, we first build a frame-wise phoneme classification network on the ASR dataset and extract the semi-supervised <linguistic features, audio> paired data from large-scale speech corpora. We then pre-train the NN based TTS acoustic model by using the semi-supervised <linguistic features, audio> pairs. Finally, we fine-tune the model with a small amount of available paired data. Experimental results show that our proposed framework enables the TTS model to generate more intelligible and natural speech with the same amount of paired training data.

On Architectures and Training for Raw Waveform Feature Extraction in ASR

Comparative Analysis of the wav2vec 2.0 Feature Extractor

ON MODULAR TRAINING OF NEURAL ACOUSTICS-TO-WORD MODEL FOR LVCSR

wav2vec and its current potential to Automatic Speech Recognition in German for the usage in Digital History: A comparative assessment of available ASR-technologies for the use in cultural heritage contexts

Multi-Span Acoustic Modelling Using Raw Waveform Signals.

Learning the Front-End Speech Feature with Raw Waveform for End-to-End Speaker Recognition

Self-Supervised Learning for Multi-Channel Neural Transducer

Efficient Utilization of Large Pre-Trained Models for Low Resource ASR

Analyzing And Improving Neural Speaker Embeddings for ASR

Exploring ASR-Based Wav2Vec2 for Automated Speech Disorder Assessment: Insights and Analysis

Unsupervised Speech Representation Learning Using WaveNet Autoencoders

Pre-training Techniques for Improving Text-to-Speech Synthesis by Automatic Speech Recognition Based Data Enhancement

A Noise-Robust Self-Supervised Pre-Training Model Based Speech Representation Learning for Automatic Speech Recognition.

Probing the Information Encoded in Neural-based Acoustic Models of Automatic Speech Recognition Systems

Wav2vec‐MoE: an Unsupervised Pre‐training and Adaptation Method for Multi‐accent ASR

Modular End-to-End Automatic Speech Recognition Framework for Acoustic-to-Word Model

On the Effect of Purely Synthetic Training Data for Different Automatic Speech Recognition Architectures

Efficient infusion of self-supervised representations in Automatic Speech Recognition

Improving Automatic Speech Recognition for Non-Native English with Transfer Learning and Language Model Decoding

wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations

Learning Architectures from an Extended Search Space for Language Modeling