Abstract:Most speech separation studies in monaural channel use only a single type of network, and the separation effect is typically not satisfactory, posing difficulties for high quality speech separation. In this study, we propose a convolutional recurrent neural network with an attention (CRNN-A) framework for speech separation, fusing advantages of two networks together. The proposed separation framework uses a convolutional neural network (CNN) as the front-end of a recurrent neural network (RNN), alleviating the problem that a sole RNN cannot effectively learn the necessary features. This framework makes use of the translation invariance provided by CNN to extract information without modifying the original signals. Within the supplemented CNN, two different convolution kernels are designed to capture information in both the time and frequency domains of the input spectrogram. After concatenating the time-domain and the frequency-domain feature maps, the feature information of speech is exploited through consecutive convolutional layers. Finally, the feature map learned from the front-end CNN is combined with the original spectrogram and is sent to the back-end RNN. Further, the attention mechanism is further incorporated, focusing on the relationship among different feature maps. The effectiveness of the proposed method is evaluated on the standard dataset MIR-1K and the results prove that the proposed method outperforms the baseline RNN and other popular speech separation methods, in terms of GNSDR (gloabl normalised source-to-distortion ratio), GSIR (global source-to-interferences ratio), and GSAR (gloabl source-to-artifacts ratio). In summary, the proposed CRNN-A framework can effectively combine the advantages of CNN and RNN, and further optimise the separation performance via the attention mechanism. The proposed framework can shed a new light on speech separation, speech enhancement, and other related fields.

Multi-layer Attention Mechanism Based Speech Separation Model.

SuperFormer: Enhanced Multi-Speaker Speech Separation Network Combining Channel and Spatial Adaptability

Dual-Path Hybrid Attention Network for Monaural Speech Separation

Speaker Extraction Network with Attention Mechanism for Speech Dialogue System

Single-Channel Speech Separation Focusing on Attention DE.

Multi-Head Self-Attention-Based Deep Clustering for Single-Channel Speech Separation

Deep Audio-Visual Speech Separation with Attention Mechanism

Multi-Stream Gated and Pyramidal Temporal Convolutional Neural Networks for Audio-Visual Speech Separation in Multi-Talker Environments

Multi-Band Pit And Model Integration For Improved Multi-Channel Speech Separation

Monaural Multi-Talker Speech Recognition With Attention Mechanism And Gated Convolutional Networks

L-Tcn with the Help of Attention Weighting for the Speech Separation Task in the Reverberation Environment

Multi-Target Ensemble Learning for Monaural Speech Separation

Multi-Speakers Speech Separation Based On Modified Attractor Points Estimation And Gmm Clustering

Multi-speaker Speech Separation under Reverberation Conditions Using Conv-Tasnet

Temporal-Spatial Neural Filter: Direction Informed End-to-End Multi-channel Target Speech Separation

A convolutional recurrent neural network with attention framework for speech separation in monaural recordings

Neural Spatial Filter: Target Speaker Speech Separation Assisted with Directional Information

A Separation and Interaction Framework for Causal Multi-Channel Speech Enhancement.

Cracking the cocktail party problem by multi-beam deep attractor network

Speaker and Direction Inferred Dual-channel Speech Separation

Monaural Speech Separation Method Based on Recurrent Attention with Parallel Branches