Abstract:We propose a novel data-driven approach to single-channel speech separation based on deep neural networks (DNNs) to directly model the highly nonlinear relationship between speech features of a mixed signal containing a target speaker and other interfering speakers. We focus our discussion on a semisupervised mode to separate speech of the target speaker from an unknown interfering speaker, which is more flexible than the conventional supervised mode with known information of both the target and interfering speakers. Two key issues are investigated. First, we propose a DNN architecture with dual outputs of the features of both the target and interfering speakers, which is shown to achieve a better generalization capability than that with output features of only the target speaker. Second, we propose using a set of multiple DNNs, each intending to be signal-noise-dependent (SND), to cope with the difficulty that one single general DNN could not well accommodate all the speaker mixing variabilities at different signal-to-noise ratio (SNR) levels. Experimental results on the speech separation challenge (SSC) data demonstrate that our proposed framework achieves better separation results than other conventional approaches in a supervised or semisupervised mode. SND-DNNs could also yield significant performance improvements over a general DNN for speech separation in low SNR cases. Furthermore, for automatic speech recognition (ASR) following speech separation, this purely front-end processing with a single set of speaker-independent ASR acoustic models, achieves a relative word error rate (WER) reduction of 11.6% over a state-of-the-art separation and recognition system where a complicated joint back-end decoding framework with multiple sets of speaker-dependent ASR acoustic models needs to be implemented. When speaker-adaptive ASR acoustic models for the target speakers are adopted for the enhanced signals, another 12.1% WER reduction over our best speaker-independent ASR system is achieved.

An Improved Supervised Speech Separation Method Based on Perceptual Weighted Deep Recurrent Neural Networks.

Speech separation based on reliable binaural cues with two-stage neural network in noisy-reverberant environments

A Regression Approach to Single-Channel Speech Separation Via High-Resolution Deep Neural Networks.

Supervised Speech Separation Based on Deep Learning: An Overview

Speech Separation Based on Signal-Noise-dependent Deep Neural Networks for Robust Speech Recognition

Speech separation of a target speaker based on deep neural networks

A Unified DNN Approach to Speaker-Dependent Simultaneous Speech Enhancement and Speech Separation in Low SNR Environments

Joint Deep Neural Network for Single-Channel Speech Separation on Masking-Based Training Targets

A Unified Speaker-Dependent Speech Separation and Enhancement System Based on Deep Neural Networks.

Speech Separation Based on Improved Deep Neural Networks with Dual Outputs of Speech Features for Both Target and Interfering Speakers

A Spectral-change-aware Loss Function for DNN-based Speech Separation.

Joint Training for Simultaneous Speech Denoising and Dereverberation with Deep Embedding Representations

LaFurca: Iterative Refined Speech Separation Based on Context-Aware Dual-Path Parallel Bi-LSTM

Noise-Aware Speech Separation with Contrastive Learning

Unsupervised Single-Channel Speech Separation Via Deep Neural Network for Different Gender Mixtures

Time-Frequency Mask Aware Bi-directional LSTM: A Deep Learning Approach for Underwater Acoustic Signal Separation

A convolutional recurrent neural network with attention framework for speech separation in monaural recordings

Improving Robustness of Deep Neural Network Acoustic Models via Speech Separation and Joint Adaptive Training

A Gender Mixture Detection Approach to Unsupervised Single-Channel Speech Separation Based on Deep Neural Networks

ADL-MVDR: All deep learning MVDR beamformer for target speech separation

Deep Learning Based Speech Separation Via NMF-Style Reconstructions.