Abstract:We propose a novel data-driven approach to single-channel speech separation based on deep neural networks (DNNs) to directly model the highly nonlinear relationship between speech features of a mixed signal containing a target speaker and other interfering speakers. We focus our discussion on a semisupervised mode to separate speech of the target speaker from an unknown interfering speaker, which is more flexible than the conventional supervised mode with known information of both the target and interfering speakers. Two key issues are investigated. First, we propose a DNN architecture with dual outputs of the features of both the target and interfering speakers, which is shown to achieve a better generalization capability than that with output features of only the target speaker. Second, we propose using a set of multiple DNNs, each intending to be signal-noise-dependent (SND), to cope with the difficulty that one single general DNN could not well accommodate all the speaker mixing variabilities at different signal-to-noise ratio (SNR) levels. Experimental results on the speech separation challenge (SSC) data demonstrate that our proposed framework achieves better separation results than other conventional approaches in a supervised or semisupervised mode. SND-DNNs could also yield significant performance improvements over a general DNN for speech separation in low SNR cases. Furthermore, for automatic speech recognition (ASR) following speech separation, this purely front-end processing with a single set of speaker-independent ASR acoustic models, achieves a relative word error rate (WER) reduction of 11.6% over a state-of-the-art separation and recognition system where a complicated joint back-end decoding framework with multiple sets of speaker-dependent ASR acoustic models needs to be implemented. When speaker-adaptive ASR acoustic models for the target speakers are adopted for the enhanced signals, another 12.1% WER reduction over our best speaker-independent ASR system is achieved.

A Hybrid Deep-Learning Approach for Single Channel HF-SSB Speech Enhancement

Audio-Visual Speech Enhancement with Deep Multi-modality Fusion

Single-channel speech enhancement using improved progressive deep neural network and masking-based harmonic regeneration

Single Channel Speech Enhancement Algorithm Based on BLSTM-DNN Bidirectional Optimized Hybrid Model

Densely Connected Multi-Stage Model with Channel Wise Subband Feature for Real-Time Speech Enhancement.

FB-MSTCN: A Full-Band Single-Channel Speech Enhancement Method Based on Multi-Scale Temporal Convolutional Network

Multi-stage Strength Estimation Network with Cross Attention for Single Channel Speech Enhancement

THLNet: two-stage heterogeneous lightweight network for monaural speech enhancement

A Dual-Channel End-to-End Speech Enhancement Method Using Complex Operations in the Time Domain

S-DCCRN: Super Wide Band DCCRN with Learnable Complex Feature for Speech Enhancement

Dual-Channel Speech Enhancement Using Neural Network Adaptive Beamforming

Deep Noise Tracking Network: A Hybrid Signal Processing/Deep Learning Approach to Speech Enhancement

FullSubNet+: Channel Attention FullSubNet with Complex Spectrograms for Speech Enhancement

Robust Automated VHF Modulation Recognition Based on Deep Convolutional Neural Networks

Monaural Speech Enhancement with Deep Residual-Dense Lattice Network and Attention Mechanism in the Time Domain

Two-Stage Single-Channel Speech Enhancement with Multi-Frame Filtering

An Attention-Based Neural Network Approach For Single Channel Speech Enhancement

Automatic Switching Technique for Speech Enhancement in HF Communications

Target Speech Signal Enhancement Based on Deep Neural Networks

A Regression Approach to Single-Channel Speech Separation Via High-Resolution Deep Neural Networks.

Speech-enhanced and Noise-aware Networks for Robust Speech Recognition