Abstract:In this paper, we develop DeepSinger, a multi-lingual multi-singer singing voice synthesis (SVS) system, which is built from scratch using singing training data mined from music websites. The pipeline of DeepSinger consists of several steps, including data crawling, singing and accompaniment separation, lyrics-to-singing alignment, data filtration, and singing modeling. Specifically, we design a lyrics-to-singing alignment model to automatically extract the duration of each phoneme in lyrics starting from coarse-grained sentence level to fine-grained phoneme level, and further design a multi-lingual multi-singer singing model based on a feed-forward Transformer to directly generate linear-spectrograms from lyrics, and synthesize voices using Griffin-Lim. DeepSinger has several advantages over previous SVS systems: 1) to the best of our knowledge, it is the first SVS system that directly mines training data from music websites, 2) the lyrics-to-singing alignment model further avoids any human efforts for alignment labeling and greatly reduces labeling cost, 3) the singing model based on a feed-forward Transformer is simple and efficient, by removing the complicated acoustic feature modeling in parametric synthesis and leveraging a reference encoder to capture the timbre of a singer from noisy singing data, and 4) it can synthesize singing voices in multiple languages and multiple singers. We evaluate DeepSinger on our mined singing dataset that consists of about 92 hours data from 89 singers on three languages (Chinese, Cantonese and English). The results demonstrate that with the singing data purely mined from the Web, DeepSinger can synthesize high-quality singing voices in terms of both pitch accuracy and voice naturalness (footnote: Our audio samples are shown in <a class="link-external link-https" href="https://speechresearch.github.io/deepsinger/" rel="external noopener nofollow">this https URL</a>.)

Towards Efficient Automated Singer Identification in Large Music Databases.

A Novel Framework for Efficient Automated Singer Identification in Large Music Databases

A Novel Singer Identification Method Using GMM-UBM

Singer Identification Using Deep Timbre Feature Learning with KNN-NET

Ensemble Model-Based Singer Classification with Proposed Vocal Segmentation

Singer identification model using data augmentation and enhanced feature conversion with hybrid feature vector and machine learning

Singer Identification for Metaverse with Timbral and Middle-Level Perceptual Features

Mixing or Extracting? Further Exploring Necessity of Music Separation for Singer Identification

Improved harmonic spectral envelope extraction for singer classification with hybridised model

NHSS: A Speech and Singing Parallel Database

UniSinger: Unified End-to-End Singing Voice Synthesis with Cross-Modality Information Matching

DeepSinger: Singing Voice Synthesis with Data Mined From the Web

Investigation of Singing Voice Separation for Singing Voice Detection in Polyphonic Music

An Effective Vocal/Non-vocal Segmentation Approach for Embedded Music Retrieve System on Mobile Phone

HANME: Hierarchical Attention Network for Singing Melody Extraction

Towards Solving The Bottleneck Of Pitch-Based Singing Voice Separation

Large-Scale Cover Song Detection in Digital Music Libraries Using Metadata, Lyrics and Audio Features

Singer separation for karaoke content generation

Jointly Recognizing Speech and Singing Voices Based on Multi-Task Audio Source Separation

Efficient Two-Layer Model Towards Cover Song Identification.

An objective evaluation method of vocal singing effect based on artificial intelligence technology