Abstract:Speech separation is the key to many speech backend tasks, like multi-speaker speech recognition. In recent years, with the development and aid of deep learning technology, many single-channel speech separation models have shown good performance in weak reverberant environment. However, with the presence of reverberation, the multi-channel speech separation model still has greater advantages. Among them, the deep neural network (DNN) based beamformers (also known as neural beamformers) have achieved significant improvements in separation quality. The current neural beamformers can’t jointly optimize beamforming layers and DNN layers when using the prior knowledge of the existing beamforming algorithms, which may make the model unable to obtain the optimal separation performance. In order to solve this problem, this paper employs a set of beamformers that uniformly sample the space as a learning module in the neural network, and the initial values of their coefficients are determined by the existing maximum directivity factor (DF) beamformer. Furthermore, to obtain beam representations of source signals when their directions are unknown, a cross-attention mechanism is introduced. The experimental results show that in the separation task with reverberation, the proposed method has better performance than the current state-of-the-art temporal neural beamformer filter-and-sum network (FasNet) and several mainstream multi-channel speech separation approaches in terms of scale-invariant signal-to-noise ratio (SI-SNR), perceptual evaluation of speech quality (PESQ) and short-time objective intelligibility measure (STOI).

A New Neural Beamformer for Multi-channel Speech Separation

DFBNet: Deep Neural Network Based Fixed Beamformer for Multi-channel Speech Separation

Locate and Beamform: Two-dimensional Locating All-neural Beamformer for Multi-channel Speech Separation

Iteratively Refined Multi-Channel Speech Separation

Neural Spatio-Temporal Beamformer for Target Speech Separation

DPTNet-based Beamforming for Speech Separation

Beamformed Feature for Learning-based Dual-channel Speech Separation

MIMO-DBnet: Multi-channel Input and Multiple Outputs DOA-aware Beamforming Network for Speech Separation

Masking-based Neural Beamformer for Multichannel Speech Enhancement

Neural Mask based Multi-channel Convolutional Beamforming for Joint Dereverberation, Echo Cancellation and Denoising

MIMO Self-Attentive RNN Beamformer for Multi-Speaker Speech Separation.

A Neural Beamspace-Domain Filter for Real-Time Multi-Channel Speech Enhancement

A Beam-TFDPRNN Based Speech Separation Method in Reverberant Environments

Binaural Reverberant Speech Separation Based on Deep Neural Networks

ADL-MVDR: All deep learning MVDR beamformer for target speech separation

Deep Learning Based Speech Beamforming

Dual-path Transformer Based Neural Beamformer for Target Speech Extraction

Deep Encoder/decoder Dual-Path Neural Network for Speech Separation in Noisy Reverberation Environments

Neural Network Adaptive Beamforming for Robust Multichannel Speech Recognition

Three-stage hybrid neural beamformer for multi-channel speech enhancement

Cracking the cocktail party problem by multi-beam deep attractor network