Abstract:Although time-domain speech separation methods have exhibited the outstanding performance in anechoic scenarios, their effectiveness is considerably reduced in the reverberant scenarios. Compared to the time-domain methods, the speech separation methods in time-frequency (T-F) domain mainly concern the structured T-F representations and have shown a great potential recently. In this paper, we propose a coarse-to-fine speech separation method in the T-F domain, which involves two steps: 1) a rough separation conducted in the coarse phase and 2) a precise extraction accomplished in the refining phase. In the coarse phase, the speech signals of all speakers are initially separated in a rough manner, resulting in some level of distortion in the estimated signals. In the refining phase, the T-F representation of each estimated signal acts as a guide to extract the residual T-F representation for the corresponding speaker, which helps to reduce the distortions caused in the coarse phase. Besides, the specially designed networks used for the coarse and refining phases are jointly trained for superior performance. Furthermore, utilizing the recurrent attention with parallel branches (RAPB) block to fully exploit the contextual information contained in the whole T-F features, the proposed model demonstrates competitive performance on clean datasets with a small number of parameters. Additionally, the proposed method shows more robustness and achieves state-of-the-art results on more realistic datasets.

Single-channel Speech Separation with Non-Negative Matrix Factorization and Factorial Conditional Random Fields

Single-Channel Speech Separation Based on Non-negative Matrix Factorization and Factorial Conditional Random Field

Layered Convolutive Nonnegative Matrix Factorization for Speech Separation

Multi-channel sound source separation based on time-frequency sparsity constraint

Transductive Convolutive Nonnegative Matrix Factorization For Speech Separation

An Adaptive Single Channel EMD-TNMF Blind Source Separation Algorithm for Both Instantaneous and Convolutive Mixed Signal

Transductive Nonnegative Matrix Factorization for Semi-Supervised High-Performance Speech Separation

An MRF-ICA based algorithm for image separation

Deep Learning Based Speech Separation Via NMF-Style Reconstructions.

Exploiting Spectro-Temporal Structures Using Nmf For Dnn-Based Supervised Speech Separation

Adaptive Beamforming Based on Interference-Plus-Noise Covariance Matrix Reconstruction for Speech Separation

NMF based speech and music separation in monaural speech recordings with sparseness and temporal continuity constraints

Deep NMF for speech separation

Monaural Singing Voice Separation By Non-Negative Matrix Partial Co-Factorization With Temporal Continuity And Sparsity Criteria

A Robust Unsupervised Method for the Single Channel Speech Separation.

Iteratively Refined Multi-Channel Speech Separation

Single channel informed signal separation using artificial‐stereophonic mixtures and exemplar‐guided matrix factor deconvolution

An Experimental Survey on Non-Negative Matrix Factorization for Single Channel Blind Source Separation

Reverberant Speech Separation with Probabilistic Time-Frequency Masking for B-format Recordings.

Single Channel Source Separation Using Filterbank and 2D Sparse Matrix Factorization

Coarse-to-fine Speech Separation Method in the Time-Frequency Domain