Abstract:Suffering from limited singing voice corpus, existing singing voice synthesis (SVS) methods that build encoder-decoder neural networks to directly generate spectrogram could lead to out-of-tune issues during the inference phase. To attenuate these issues, this paper presents a novel acoustic model with independent pitch encoder and phoneme encoder, which disentangles the phoneme and pitch information from music score to fully utilize the corpus. Specifically, according to equal temperament theory, the pitch encoder is constrained by a pitch metric loss that maps distances between adjacent input pitches into corresponding frequency multiples between the encoder outputs. For the phoneme encoder, based on the analysis that same phonemes corresponding to varying pitches can produce similar pronunciations, this encoder is followed by an adversarially trained pitch classifier to enforce the identical phonemes with different pitches mapping into the same phoneme feature space. By these means, the sparse phonemes and pitches in original input spaces can be transformed into more compact feature spaces respectively, where the same elements cluster closely and cooperate mutually to enhance synthesis quality. Then, the outputs of the two encoders are summed together to pass through the following decoder in the acoustic model. Experimental results indicate that the proposed approaches can characterize intrinsic structure between pitch inputs to obtain better pitch synthesis accuracy and achieve superior singing synthesis performance against the advanced baseline system.

A Pitch Correction System Based on Spectral Modeling Synthesis Technique

CONTUNER: Singing Voice Beautifying with Pitch and Expressiveness Condition

A Cross-Modal Approach for Karaoke Artifacts Correction

Pitch Preservation In Singing Voice Synthesis

Speech Enhancement Based On Analysis Synthesis Framework With Improved Pitch Estimation And Spectral Envelope Enhancement

An initial research: Towards accurate pitch extraction for speech synthesis based on BLSTM

Toward Expressive Singing Voice Correction: On Perceptual Validity of Evaluation Metrics for Vocal Melody Extraction

Pitch-and-Spectrum-Aware Singing Quality Assessment with Bias Correction and Model Fusion

Research on real-time detection technology of Chinese voiced speech pitch

Pitch Models of Mandarin Text-to-speech

Pitch-Scaled Spectrum Based Excitation Model for HMM-based Speech Synthesis

Investigation of the Spectral Envelope Estimation Vocoder and Improved Pitch Estimation Based on the Sinusoidal Speech Model

A Robust and Low Computational Cost Pitch Estimation Method

A New Robust Pitch Determination Algorithm for Telephone Speech

Speech Enhancement Based on Analysis–Synthesis Framework with Improved Parameter Domain Enhancement

Error Correction Scheme for Vocoder Pitch

Mandarin Singing Voice Synthesis Based on Harmonic Plus Noise Model and Singing Expression Analysis

EditSinger: Zero-Shot Text-Based Singing Voice Editing System with Diverse Prosody Modeling

Expressive Singing Synthesis Using Local Style Token and Dual-path Pitch Encoder

SPA-SVC: Self-supervised Pitch Augmentation for Singing Voice Conversion

Using Pitch Feature Matching to Design a Music Tutoring System Based on Deep Learning