Abstract:The traditional vocoders have the advantages of high synthesis efficiency, strong interpretability, and speech editability, while the neural vocoders have the advantage of high synthesis quality. To combine the advantages of two vocoders, inspired by the traditional deterministic plus stochastic model, this paper proposes a novel neural vocoder named NeuralDPS which can retain high speech quality and acquire high synthesis efficiency and noise controllability. Firstly, this framework contains four modules: a deterministic source module, a stochastic source module, a neural V/UV decision module and a neural filter module. The input required by the vocoder is just the spectral parameter, which avoids the error caused by estimating additional parameters, such as F0. Secondly, to solve the problem that different frequency bands may have different proportions of deterministic components and stochastic components, a multiband excitation strategy is used to generate a more accurate excitation signal and reduce the neural filter's burden. Thirdly, a method to control noise components of speech is proposed. In this way, the signal-to-noise ratio (SNR) of speech can be adjusted easily. Objective and subjective experimental results show that our proposed NeuralDPS vocoder can obtain similar performance with the WaveNet and it generates waveforms at least 280 times faster than the WaveNet vocoder. It is also 28% faster than WaveGAN's synthesis efficiency on a single CPU core. We have also verified through experiments that this method can effectively control the noise components in the predicted speech and adjust the SNR of speech. Examples of generated speech can be found at <a class="link-external link-https" href="https://hairuo55.github.io/NeuralDPS" rel="external noopener nofollow">this https URL</a>.

A Fast High-Fidelity Source-Filter Vocoder with Lightweight Neural Modules.

Neural Homomorphic Vocoder.

NeuralDPS: Neural Deterministic Plus Stochastic Model with Multiband Excitation for Noise-Controllable Waveform Generation

ESTVocoder: An Excitation-Spectral-Transformed Neural Vocoder Conditioned on Mel Spectrogram

Ultra-lightweight Neural Differential DSP Vocoder For High Quality Speech Synthesis

Vocos: Closing the gap between time-domain and Fourier-based neural vocoders for high-quality audio synthesis

WOLONet: Wave Outlooker for Efficient and High Fidelity Speech Synthesis

A Neural Vocoder with Hierarchical Generation of Amplitude and Phase Spectra for Statistical Parametric Speech Synthesis

Source-Filter-Based Generative Adversarial Neural Vocoder for High Fidelity Speech Synthesis

A Neural Denoising Vocoder for Clean Waveform Generation from Noisy Mel-Spectrogram based on Amplitude and Phase Predictions

FreeV: Free Lunch For Vocoders Through Pseudo Inversed Mel Filter

Samplernn-Based Neural Vocoder for Statistical Parametric Speech Synthesis.

Puffin: pitch-synchronous neural waveform generation for fullband speech on modest devices

FastFit: Towards Real-Time Iterative Neural Vocoder by Replacing U-Net Encoder With Multiple STFTs

MusicHiFi: Fast High-Fidelity Stereo Vocoding

LA-VocE: Low-SNR Audio-visual Speech Enhancement using Neural Vocoders

Hierarchical RNNs for Waveform-Level Speech Synthesis

A High Fidelity and Low Complexity Neural Audio Coding

BiVocoder: A Bidirectional Neural Vocoder Integrating Feature Extraction and Waveform Generation

DeepGAN: A Fast and High-Quality Time-Domain-based Neural Vocoder for Low-Resource Scenarios

HiFTNet: A Fast High-Quality Neural Vocoder with Harmonic-plus-Noise Filter and Inverse Short Time Fourier Transform