Abstract:In this article, we propose an end-to-end post-filter method with deep attention fusion features for monaural speaker-independent speech separation. At first, a time-frequency domain speech separation method is applied as the pre-separation stage. The aim of pre-separation stage is to separate the mixture preliminarily. Although this stage can separate the mixture, it still contains the residual interference. In order to enhance the pre-separated speech and improve the separation performance further, the end-to-end post-filter (E2EPF) with deep attention fusion features is proposed. The E2EPF can make full use of the prior knowledge of the pre-separated speech, which contributes to speech separation. It is a fully convolutional speech separation network and uses the waveform as the input features. Firstly, the 1-D convolutional layer is utilized to extract the deep representation features for the mixture and pre-separated signals in the time domain. Secondly, to pay more attention to the outputs of the pre-separation stage, an attention module is applied to acquire deep attention fusion features, which are extracted by computing the similarity between the mixture and the pre-separated speech. These deep attention fusion features are conducive to reduce the interference and enhance the pre-separated speech. Finally, these features are sent to the post-filter to estimate each target signals. Experimental results on the WSJ0-2mix dataset show that the proposed method outperforms the state-of-the-art speech separation method. Compared with the pre-separation method, our proposed method can acquire 64.1%, 60.2%, 25.6% and 7.5% relative improvements in scale-invariant source-to-noise ratio (SI-SNR), the signal-to-distortion ratio (SDR), the perceptual evaluation of speech quality (PESQ) and the short-time objective intelligibility (STOI) measures, respectively.

MAF-Net: Multidimensional Attention Fusion Network for Multichannel Speech Separation

Audio-Visual Speech Enhancement with Deep Multi-modality Fusion

Gated Recurrent Fusion of Spatial and Spectral Features for Multi-Channel Speech Separation with Deep Embedding Representations.

Deep Attention Fusion Feature for Speech Separation with End-to-End Post-filter Method

SpatialNet: Extensively Learning Spatial Information for Multichannel Joint Speech Separation, Denoising and Dereverberation

Temporal-Spatial Neural Filter: Direction Informed End-to-End Multi-channel Target Speech Separation

MAFusion: Multiscale Attention Network for Infrared and Visible Image Fusion

MSAF: Multimodal Split Attention Fusion

End-to-End Post-Filter for Speech Separation with Deep Attention Fusion Features

An Efficient Speech Separation Network Based on Recurrent Fusion Dilated Convolution and Channel Attention

MSFNet: Multi-Scale Fusion Network for Brain-Controlled Speaker Extraction

Joint Deep Neural Network for Single-Channel Speech Separation on Masking-Based Training Targets

A New Neural Beamformer for Multi-channel Speech Separation

AMFFCN: Attentional Multi-layer Feature Fusion Convolution Network for Audio-visual Speech Enhancement

Improving Visual Speech Enhancement Network by Learning Audio-visual Affinity with Multi-head Attention

Multi-channel Multi-frame ADL-MVDR for Target Speech Separation

Multi-stage music separation network with dual-branch attention and hybrid convolution

A Feature Integration Network for Multi-Channel Speech Enhancement

SCANet: A Self- and Cross-Attention Network for Audio-Visual Speech Separation

A time-frequency fusion model for multi-channel speech enhancement

The Safety of Drug Therapy in Children.