Abstract:Monaural source separation (MSS) aims to extract and reconstruct different sources from a single-channel mixture, which could facilitate a variety of applications such as chord recognition, pitch estimation and automatic transcription. In this paper, we study the problem of separating vocals and instruments from monaural music mixture. Existing works for monaural source separation either utilize linear and shallow models (e.g., non-negative matrix factorization), or do not explicitly address the coupling and tangling of multiple sources in original input signals, hence they do not perform satisfactorily in real-world scenarios. To overcome the above limitations, we propose a novel end-to-end framework for monaural music mixture separation called Deep Representation-Decoupling Neural Networks (DRDNN). DRDNN takes advantages of both traditional signal processing methods and popular deep learning models. For each input of music mixture, DRDNN converts it to a two-dimensional time-frequency spectrogram using short-time Fourier transform (STFT), followed by stacked convolutional neural networks (CNN) layers and long-short term memory (LSTM) layers to extract more condensed features. Afterwards, DRDNN utilizes a decoupling component, which consists of a group of multi-layer perceptrons (MLP), to decouple the features further into different separated sources. The design of decoupling component in DRDNN produces purified single-source signals for subsequent full-size restoration, and can significantly improve the performance of final separation. Through extensive experiments on real-world dataset, we prove that DRDNN outperforms state-of-the-art baselines in the task of monaural music mixture separation and reconstruction.

Multi-Stage Non-Negative Matrix Factorization for Monaural Singing Voice Separation

Separation of Singing Voice Using Nonnegative Matrix Partial Co-Factorization for Singer Identification

Towards Solving The Bottleneck Of Pitch-Based Singing Voice Separation

A Local Discontinuity Based Approach for Monaural Singing Voice Separation from Accompanying Music with Multi-Stage Non-Negative Matrix Factorization

Unsupervised Single-Channel Singing Voice Separation with Weighted Robust Principal Component Analysis Based on Gammatone Auditory Filterbank and Vocal Activity Detection

Latent time-frequency component analysis: A novel pitch-based approach for singing voice separation

Investigation of Singing Voice Separation for Singing Voice Detection in Polyphonic Music

Combining HMM-based melody extraction and NMF-based soft masking for separating voice and accompaniment from monaural audio

Singing Voice Separation and Vocal F0 Estimation based on Mutual Combination of Robust Principal Component Analysis and Subharmonic Summation

Multi-Band Multi-Resolution Fully Convolutional Neural Networks for Singing Voice Separation

Comparison for Improvements of Singing Voice Detection System Based on Vocal Separation

NMF based speech and music separation in monaural speech recordings with sparseness and temporal continuity constraints

Audiovisual Singing Voice Separation

Singer identification based on computational auditory scene analysis and missing feature methods

Singer separation for karaoke content generation

Depthwise Separable Convolutions Versus Recurrent Neural Networks for Monaural Singing Voice Separation

Mixing or Extracting? Further Exploring Necessity of Music Separation for Singer Identification

Jointly Detecting and Separating Singing Voice: A Multi-Task Approach

Soft constrained leading voice separation with music score guidance

Deep Representation-Decoupling Neural Networks for Monaural Music Mixture Separation

Spectral Mapping of Singing Voices: U-Net-Assisted Vocal Segmentation