Abstract:Most speech separation studies in monaural channel use only a single type of network, and the separation effect is typically not satisfactory, posing difficulties for high quality speech separation. In this study, we propose a convolutional recurrent neural network with an attention (CRNN-A) framework for speech separation, fusing advantages of two networks together. The proposed separation framework uses a convolutional neural network (CNN) as the front-end of a recurrent neural network (RNN), alleviating the problem that a sole RNN cannot effectively learn the necessary features. This framework makes use of the translation invariance provided by CNN to extract information without modifying the original signals. Within the supplemented CNN, two different convolution kernels are designed to capture information in both the time and frequency domains of the input spectrogram. After concatenating the time-domain and the frequency-domain feature maps, the feature information of speech is exploited through consecutive convolutional layers. Finally, the feature map learned from the front-end CNN is combined with the original spectrogram and is sent to the back-end RNN. Further, the attention mechanism is further incorporated, focusing on the relationship among different feature maps. The effectiveness of the proposed method is evaluated on the standard dataset MIR-1K and the results prove that the proposed method outperforms the baseline RNN and other popular speech separation methods, in terms of GNSDR (gloabl normalised source-to-distortion ratio), GSIR (global source-to-interferences ratio), and GSAR (gloabl source-to-artifacts ratio). In summary, the proposed CRNN-A framework can effectively combine the advantages of CNN and RNN, and further optimise the separation performance via the attention mechanism. The proposed framework can shed a new light on speech separation, speech enhancement, and other related fields.

Multi-stage music separation network with dual-branch attention and hybrid convolution

Voice and accompaniment separation in music using self-attention convolutional neural network

Multi-Band Multi-Resolution Fully Convolutional Neural Networks for Singing Voice Separation

Multi-branch Learning for Noisy and Reverberant Monaural Speech Separation

A convolutional recurrent neural network with attention framework for speech separation in monaural recordings

Gated Recurrent Fusion of Spatial and Spectral Features for Multi-Channel Speech Separation with Deep Embedding Representations.

Hierarchic Temporal Convolutional Network With Cross-Domain Encoder for Music Source Separation

Double Branches and Stages Neural Network for Joint Acoustic Echo and Noise Suppression

Multi-Dimensional and Multi-Scale Modeling for Speech Separation Optimized by Discriminative Learning

Depthwise Separable Convolutions Versus Recurrent Neural Networks for Monaural Singing Voice Separation

Efficient Monaural Speech Separation with Multiscale Time-Delay Sampling

Jointly Recognizing Speech and Singing Voices Based on Multi-Task Audio Source Separation

Separate-to-Recognize: Joint Multi-target Speech Separation and Speech Recognition for Speaker-attributed ASR

Joint Deep Neural Network for Single-Channel Speech Separation on Masking-Based Training Targets

Multi-scale temporal-frequency attention for music source separation

Multi-Stage Non-Negative Matrix Factorization for Monaural Singing Voice Separation

A Two-Stage Band-Split Mamba-2 Network For Music Separation

A neural harmonic-aware network with gated attentive fusion for singing melody extraction

Dual-Branch Attention-In-Attention Transformer for Single-Channel Speech Enhancement

An Efficient Speech Separation Network Based on Recurrent Fusion Dilated Convolution and Channel Attention

On End-to-end Multi-channel Time Domain Speech Separation in Reverberant Environments