Abstract:Speech separation is the key to many speech backend tasks, like multi-speaker speech recognition. In recent years, with the development and aid of deep learning technology, many single-channel speech separation models have shown good performance in weak reverberant environment. However, with the presence of reverberation, the multi-channel speech separation model still has greater advantages. Among them, the deep neural network (DNN) based beamformers (also known as neural beamformers) have achieved significant improvements in separation quality. The current neural beamformers can’t jointly optimize beamforming layers and DNN layers when using the prior knowledge of the existing beamforming algorithms, which may make the model unable to obtain the optimal separation performance. In order to solve this problem, this paper employs a set of beamformers that uniformly sample the space as a learning module in the neural network, and the initial values of their coefficients are determined by the existing maximum directivity factor (DF) beamformer. Furthermore, to obtain beam representations of source signals when their directions are unknown, a cross-attention mechanism is introduced. The experimental results show that in the separation task with reverberation, the proposed method has better performance than the current state-of-the-art temporal neural beamformer filter-and-sum network (FasNet) and several mainstream multi-channel speech separation approaches in terms of scale-invariant signal-to-noise ratio (SI-SNR), perceptual evaluation of speech quality (PESQ) and short-time objective intelligibility measure (STOI).

A Beam-TFDPRNN Based Speech Separation Method in Reverberant Environments

DPTNet-based Beamforming for Speech Separation

Iteratively Refined Multi-Channel Speech Separation

Beam-TasNet: Time-domain Audio Separation Network Meets Frequency-domain Beamformer

DFBNet: Deep Neural Network Based Fixed Beamformer for Multi-channel Speech Separation

A New Neural Beamformer for Multi-channel Speech Separation

Beam-Guided TasNet: an Iterative Speech Separation Framework with Multi-Channel Output

Neural Spatio-Temporal Beamformer for Target Speech Separation

Generalized Spatio-Temporal RNN Beamformer for Target Speech Separation

ADL-MVDR: All deep learning MVDR beamformer for target speech separation

Localization Based Stereo Speech Separation Using Deep Networks.

Generalized RNN Beamformer for Target Speech Separation.

Reverberant Speech Separation with Probabilistic Time-Frequency Masking for B-format Recordings.

Localization Based Stereo Speech Source Separation Using Probabilistic Time-Frequency Masking and Deep Neural Networks

Beamformed Feature for Learning-based Dual-channel Speech Separation

Coarse-to-fine Speech Separation Method in the Time-Frequency Domain

Dual-Channel Speech Enhancement Using Neural Network Adaptive Beamforming

A reverberation-time-aware DNN approach leveraging spatial information for microphone array dereverberation

MIMO Self-Attentive RNN Beamformer for Multi-Speaker Speech Separation.

On End-to-end Multi-channel Time Domain Speech Separation in Reverberant Environments

Multi-channel Speech Enhancement Based on the MVDR Beamformer and Postfilter.