Abstract:This paper presents a new spectral modeling method for statistical parametric speech synthesis. In the conventional methods, high-level spectral parameters, such as mel-cepstra or line spectral pairs, are adopted as the features for hidden Markov model (HMM)-based parametric speech synthesis. Our proposed method described in this paper improves the conventional method in two ways. First, distributions of low-level, un-transformed spectral envelopes (extracted by the STRAIGHT vocoder) are used as the parameters for synthesis. Second, instead of using single Gaussian distribution, we adopt the graphical models with multiple hidden variables, including restricted Boltzmann machines (RBM) and deep belief networks (DBN), to represent the distribution of the low-level spectral envelopes at each HMM state. At the synthesis time, the spectral envelopes are predicted from the RBM-HMMs or the DBN-HMMs of the input sentence following the maximum output probability parameter generation criterion with the constraints of the dynamic features. A Gaussian approximation is applied to the marginal distribution of the visible stochastic variables in the RBM or DBN at each HMM state in order to achieve a closed-form solution to the parameter generation problem. Our experimental results show that both RBM-HMM and DBN-HMM are able to generate spectral envelope parameter sequences better than the conventional Gaussian-HMM with superior generalization capabilities and that DBN-HMM and RBM-HMM perform similarly due possibly to the use of Gaussian approximation. As a result, our proposed method can significantly alleviate the over-smoothing effect and improve the naturalness of the conventional HMM-based speech synthesis system using mel-cepstra.

Using conditional restricted Boltzmann machines for spectral envelope modeling in speech bandwidth extension.

Modeling Spectral Envelopes Using Deep Conditional Restricted Boltzmann Machines for Statistical Parametric Speech Synthesis.

Restoring High Frequency Spectral Envelopes Using Neural Networks For Speech Bandwidth Extension

Modeling spectral envelopes using restricted Boltzmann machines for statistical parametric speech synthesis

Voice Conversion Using Conditional Restricted Boltzmann Machine

Modeling Spectral Envelopes Using Restricted Boltzmann Machines and Deep Belief Networks for Statistical Parametric Speech Synthesis

Speech Bandwidth Extension Using Bottleneck Features and Deep Recurrent Neural Networks.

Speech neuromuscular decoding based on spectrogram images using conformal predictors with Bi-LSTM.

Waveform Modeling and Generation Using Hierarchical Recurrent Neural Networks for Speech Bandwidth Extension

A Novel Research to Artificial Bandwidth Extension Based on Deep BLSTM Recurrent Neural Networks and Exemplar-Based Sparse Representation.

A Novel Unified Framework for Speech Enhancement and Bandwidth Extension Based on Jointly Trained Neural Networks

Spectral Modeling Using Neural Autoregressive Distribution Estimators for Statistical Parametric Speech Synthesis

Restricted Boltzmann Machine Based Spectrum Modeling and Unit Selection Speech Synthesis Method

Complex-Valued Restricted Boltzmann Machine for Direct Speech Parameterization from Complex Spectra

Joint Spectral Distribution Modeling Using Restricted Boltzmann Machines For Voice Conversion

Voice Conversion Using Deep Neural Networks with Layer-Wise Generative Training

Waveform Modeling Using Stacked Dilated Convolutional Neural Networks for Speech Bandwidth Extension.

Using bidirectional associative memories for joint spectral envelope modeling in voice conversion

Time-Domain Neural Network Approach for Speech Bandwidth Extension.

DSP-informed bandwidth extension using locally-conditioned excitation and linear time-varying filter subnetworks

Voice Conversion Using Generative Trained Deep Neural Networks with Multiple Frame Spectral Envelopes