Abstract:We propose a novel data-driven approach to single-channel speech separation based on deep neural networks (DNNs) to directly model the highly nonlinear relationship between speech features of a mixed signal containing a target speaker and other interfering speakers. We focus our discussion on a semisupervised mode to separate speech of the target speaker from an unknown interfering speaker, which is more flexible than the conventional supervised mode with known information of both the target and interfering speakers. Two key issues are investigated. First, we propose a DNN architecture with dual outputs of the features of both the target and interfering speakers, which is shown to achieve a better generalization capability than that with output features of only the target speaker. Second, we propose using a set of multiple DNNs, each intending to be signal-noise-dependent (SND), to cope with the difficulty that one single general DNN could not well accommodate all the speaker mixing variabilities at different signal-to-noise ratio (SNR) levels. Experimental results on the speech separation challenge (SSC) data demonstrate that our proposed framework achieves better separation results than other conventional approaches in a supervised or semisupervised mode. SND-DNNs could also yield significant performance improvements over a general DNN for speech separation in low SNR cases. Furthermore, for automatic speech recognition (ASR) following speech separation, this purely front-end processing with a single set of speaker-independent ASR acoustic models, achieves a relative word error rate (WER) reduction of 11.6% over a state-of-the-art separation and recognition system where a complicated joint back-end decoding framework with multiple sets of speaker-dependent ASR acoustic models needs to be implemented. When speaker-adaptive ASR acoustic models for the target speakers are adopted for the enhanced signals, another 12.1% WER reduction over our best speaker-independent ASR system is achieved.

Multi-mode Study of Deep Learning Applications in Acoustic Signal Processing

Deep Neural Network Based Environment Sound Classification and Its Implementation on Hearing Aid App

An Investigation of High-Resolution Modeling Units of Deep Neural Networks for Acoustic Scene Classification

Toward a Better Understanding of Deep Neural Network Based Acoustic Modelling: An Empirical Investigation

A Unified Speaker-Dependent Speech Separation and Enhancement System Based on Deep Neural Networks.

Ensemble Of Deep Neural Networks For Acoustic Scene Classification

Deep Neural Network Based Noised Asian Speech Enhancement and Its Implementation on a Hearing Aid App.

Deep Learning Applied to Dereverberation and Sound Event Classification in Reverberant Environments

An Investigation on Multiscale Normalised Deep Scattering Spectrum with Deep Residual Network for Acoustic Scene Classification

Hierarchical learning for DNN-based acoustic scene classification

A Deep Neural Network Approach to Acoustic Source Localization in a Shallow Water Tank Experiment

Acoustic Emission Signal Classification Using Feature Analysis and Deep Learning Neural Network

A Unified DNN Approach to Speaker-Dependent Simultaneous Speech Enhancement and Speech Separation in Low SNR Environments

AN ENSEMBLE SYSTEM FOR DOMESTIC ACTIVITY RECOGNITION Technical Report

State-Clustering Based Multiple Deep Neural Networks Modeling Approach for Speech Recognition

A Comparison of deep learning methods for environmental sound

Constrained Learned Feature Extraction for Acoustic Scene Classification

Recent Progresses in Deep Learning based Acoustic Models (Updated)

Multi-Channel Automatic Speech Recognition Using Deep Complex Unet

Sound source localization method based time-domain signal feature using deep learning

A Regression Approach to Single-Channel Speech Separation Via High-Resolution Deep Neural Networks.