Abstract:Speech separation is the key to many speech backend tasks, like multi-speaker speech recognition. In recent years, with the development and aid of deep learning technology, many single-channel speech separation models have shown good performance in weak reverberant environment. However, with the presence of reverberation, the multi-channel speech separation model still has greater advantages. Among them, the deep neural network (DNN) based beamformers (also known as neural beamformers) have achieved significant improvements in separation quality. The current neural beamformers can’t jointly optimize beamforming layers and DNN layers when using the prior knowledge of the existing beamforming algorithms, which may make the model unable to obtain the optimal separation performance. In order to solve this problem, this paper employs a set of beamformers that uniformly sample the space as a learning module in the neural network, and the initial values of their coefficients are determined by the existing maximum directivity factor (DF) beamformer. Furthermore, to obtain beam representations of source signals when their directions are unknown, a cross-attention mechanism is introduced. The experimental results show that in the separation task with reverberation, the proposed method has better performance than the current state-of-the-art temporal neural beamformer filter-and-sum network (FasNet) and several mainstream multi-channel speech separation approaches in terms of scale-invariant signal-to-noise ratio (SI-SNR), perceptual evaluation of speech quality (PESQ) and short-time objective intelligibility measure (STOI).

Masking-based Neural Beamformer for Multichannel Speech Enhancement

Robust Mask Estimation by Integrating Neural Network-Based and Clustering-Based Approaches for Adaptive Acoustic Beamforming.

Neural Mask based Multi-channel Convolutional Beamforming for Joint Dereverberation, Echo Cancellation and Denoising

Beamforming-based Speech Enhancement Based on Optimal Ratio Mask

A Neural Beamspace-Domain Filter for Real-Time Multi-Channel Speech Enhancement

Deep Learning Based Speech Beamforming

Multi-channel Speech Enhancement Based on the MVDR Beamformer and Postfilter.

Robust Beamforming for Speech Recognition Using DNN-Based Time-Frequency Masks Estimation.

A New Neural Beamformer for Multi-channel Speech Separation

Speech Enhancement Integrating the MVDR Beamforming and T-F Masking

Neural Network Adaptive Beamforming for Robust Multichannel Speech Recognition

Neural Spatio-Temporal Beamformer for Target Speech Separation

Attention-Based Beamformer For Multi-Channel Speech Enhancement

Deep Long Short-Term Memory Adaptive Beamforming Networks For Multichannel Robust Speech Recognition

An Implementaion of the CNN-Based MVDR Beamforming For Speech Enhancement.

Single-channel speech enhancement using improved progressive deep neural network and masking-based harmonic regeneration

Speech Enhancement Based on Beamforming and Post-Filtering by Combining Phase Information

Dual-Channel Speech Enhancement Using Neural Network Adaptive Beamforming

A unified multichannel far-field speech recognition system: combining neural beamforming with attention based end-to-end model

Two-stage unet with channel and temporal-frequency attention for multi-channel speech enhancement

Joint Training Of Complex Ratio Mask Based Beamformer And Acoustic Model For Noise Robust Asr