Abstract:Speech separation is the key to many speech backend tasks, like multi-speaker speech recognition. In recent years, with the development and aid of deep learning technology, many single-channel speech separation models have shown good performance in weak reverberant environment. However, with the presence of reverberation, the multi-channel speech separation model still has greater advantages. Among them, the deep neural network (DNN) based beamformers (also known as neural beamformers) have achieved significant improvements in separation quality. The current neural beamformers can’t jointly optimize beamforming layers and DNN layers when using the prior knowledge of the existing beamforming algorithms, which may make the model unable to obtain the optimal separation performance. In order to solve this problem, this paper employs a set of beamformers that uniformly sample the space as a learning module in the neural network, and the initial values of their coefficients are determined by the existing maximum directivity factor (DF) beamformer. Furthermore, to obtain beam representations of source signals when their directions are unknown, a cross-attention mechanism is introduced. The experimental results show that in the separation task with reverberation, the proposed method has better performance than the current state-of-the-art temporal neural beamformer filter-and-sum network (FasNet) and several mainstream multi-channel speech separation approaches in terms of scale-invariant signal-to-noise ratio (SI-SNR), perceptual evaluation of speech quality (PESQ) and short-time objective intelligibility measure (STOI).

DFBNet: Deep Neural Network Based Fixed Beamformer for Multi-channel Speech Separation

A New Neural Beamformer for Multi-channel Speech Separation

DPTNet-based Beamforming for Speech Separation

A Beam-TFDPRNN Based Speech Separation Method in Reverberant Environments

Locate and Beamform: Two-dimensional Locating All-neural Beamformer for Multi-channel Speech Separation

MIMO-DBnet: Multi-channel Input and Multiple Outputs DOA-aware Beamforming Network for Speech Separation

Iteratively Refined Multi-Channel Speech Separation

Beamformed Feature for Learning-based Dual-channel Speech Separation

Deep Neural Network Based Audio Source Separation

Binaural Reverberant Speech Separation Based on Deep Neural Networks

DE-DPCTnet: Deep Encoder Dual-path Convolutional Transformer Network for Multi-channel Speech Separation

Neural Spatio-Temporal Beamformer for Target Speech Separation

A Binaural Heterophasic Adaptive Beamformer and Its Deep Learning Assisted Implementation

Localization Based Stereo Speech Separation Using Deep Networks.

Deep Neural Mel-Subband Beamformer for In-Car Speech Separation

Generalized RNN Beamformer for Target Speech Separation.

Deep Learning Based Speech Beamforming

MIMO Self-Attentive RNN Beamformer for Multi-Speaker Speech Separation.

Beamforming and Deep Models Integrated Multi-talker Speech Separation

Explainable DNN-based Beamformer with Postfilter