Abstract:This paper presents a new spectral envelope conversion method using deep neural networks (DNNs). The conventional joint density Gaussian mixture model (JDGMM) based spectral conversion methods perform stably and effectively. However, the speech generated by these methods suffer severe quality degradation due to the following two factors: 1) inadequacy of JDGMM in modeling the distribution of spectral features as well as the non-linear mapping relationship between the source and target speakers, 2) spectral detail loss caused by the use of high-level spectral features such as mel-cepstra. Previously, we have proposed to use the mixture of restricted Boltzmann machines (MoRBM) and the mixture of Gaussian bidirectional associative memories (MoGBAM) to cope with these problems. In this paper, we propose to use a DNN to construct a global non-linear mapping relationship between the spectral envelopes of two speakers. The proposed DNN is generatively trained by cascading two RBMs, which model the distributions of spectral envelopes of source and target speakers respectively, using a Bernoulli BAM (BBAM). Therefore, the proposed training method takes the advantage of the strong modeling ability of RBMs in modeling the distribution of spectral envelopes and the superiority of BAMs in deriving the conditional distributions for conversion. Careful comparisons and analysis among the proposed method and some conventional methods are presented in this paper. The subjective results show that the proposed method can significantly improve the performance in terms of both similarity and naturalness compared to conventional methods.

Restoring High Frequency Spectral Envelopes Using Neural Networks For Speech Bandwidth Extension

Speech Bandwidth Extension Using Bottleneck Features and Deep Recurrent Neural Networks.

Speech Bandwidth Extension Based on Restricted Boltzmann Machines

Recurrent neural network for spectral mapping in speech bandwidth extension

Speech Bandwidth Expansion Based on Deep Neural Networks

Waveform Modeling and Generation Using Hierarchical Recurrent Neural Networks for Speech Bandwidth Extension

Using conditional restricted Boltzmann machines for spectral envelope modeling in speech bandwidth extension.

Speech bandwidth extension supported by temporal information

Spectral Envelope Estimation Used for Audio Bandwidth Extension Based on RBF Neural Network

Audio Bandwidth Extension Using Ensemble of Recurrent Neural Networks

A Novel Unified Framework for Speech Enhancement and Bandwidth Extension Based on Jointly Trained Neural Networks

Speech Bandwidth Extension Using Recurrent Temporal Restricted Boltzmann Machines

Waveform Modeling Using Stacked Dilated Convolutional Neural Networks for Speech Bandwidth Extension.

Audio bandwidth extension based on RBF neural network

A Novel Method of Artificial Bandwidth Extension Using Deep Architecture.

BAE-Net: A Low complexity and high fidelity Bandwidth-Adaptive neural network for speech super-resolution

Vector Quantized Diffusion Model Based Speech Bandwidth Extension

Time-Domain Neural Network Approach for Speech Bandwidth Extension.

A Novel Research to Artificial Bandwidth Extension Based on Deep BLSTM Recurrent Neural Networks and Exemplar-Based Sparse Representation.

Voice Conversion Using Deep Neural Networks with Layer-Wise Generative Training

Blind Bandwidth Extension Of Audio Signals Based On Non-Linear Prediction And Hidden Markov Model