Abstract:Deep generative models have achieved significant progress in speech synthesis to date, while high-fidelity singing voice synthesis is still an open problem for its long continuous pronunciation, rich high-frequency parts, and strong expressiveness. Existing neural vocoders designed for text-to-speech cannot directly be applied to singing voice synthesis because they result in glitches and poor high-frequency reconstruction. In this work, we propose SingGAN, a generative adversarial network designed for high-fidelity singing voice synthesis. Specifically, 1) to alleviate the glitch problem in the generated samples, we propose source excitation with the adaptive feature learning filters to expand the receptive field patterns and stabilize long continuous signal generation; and 2) SingGAN introduces global and local discriminators at different scales to enrich low-frequency details and promote high-frequency reconstruction; and 3) To improve the training efficiency, SingGAN includes auxiliary spectrogram losses and sub-band feature matching penalty loss. To the best of our knowledge, SingGAN is the first work designed toward high-fidelity singing voice vocoding. Our evaluation of SingGAN demonstrates the state-of-the-art results with higher-quality (MOS 4.05) samples. Also, SingGAN enables a sample speed of 50x faster than real-time on a single NVIDIA 2080Ti GPU. We further show that SingGAN generalizes well to the mel-spectrogram inversion of unseen singers, and the end-to-end singing voice synthesis system SingGAN-SVS enjoys a two-stage pipeline to transform the music scores into expressive singing voices.

A Practical Singing Voice Detection System Based on GRU-RNN

Research On Singing Voice Detection Based On A Long-Term Recurrent Convolutional Network With Vocal Separation And Temporal Smoothing

Comparison for Improvements of Singing Voice Detection System Based on Vocal Separation

G-RNN-GAN for Singing Voice Separation

Investigation of Singing Voice Separation for Singing Voice Detection in Polyphonic Music

Gen-Res-Net: A Novel Generative Model for Singing Voice Separation

PrimaDNN': A Characteristics-aware DNN Customization for Singing Technique Detection

Principle Analysis of Voiceprint Identification and Authentication System Based on LSTM

Multi-Band Multi-Resolution Fully Convolutional Neural Networks for Singing Voice Separation

Jointly Detecting and Separating Singing Voice: A Multi-Task Approach

Singing Voice Graph Modeling for SingFake Detection

Singing Voice Detection Using Multi-Feature Deep Fusion with CNN

Automatic Singing Voice Detection for Songs Based on a Dynamic Slicing Method with Beat

Deep Audio-Visual Singing Voice Transcription based on Self-Supervised Learning Models

ByteSing: A Chinese Singing Voice Synthesis System Using Duration Allocated Encoder-Decoder Acoustic Models and WaveRNN Vocoders

SingGAN: Generative Adversarial Network for High-Fidelity Singing Voice Generation

Leveraging Laryngograph Data for Robust Voicing Detection in Speech

A Survey on Recent Deep Learning-driven Singing Voice Synthesis Systems

Spectral Mapping of Singing Voices: U-Net-Assisted Vocal Segmentation

Neural Vocoder Feature Estimation for Dry Singing Voice Separation

Deep Learning Approaches in Topics of Singing Information Processing