Abstract:Speech separation is the key to many speech backend tasks, like multi-speaker speech recognition. In recent years, with the development and aid of deep learning technology, many single-channel speech separation models have shown good performance in weak reverberant environment. However, with the presence of reverberation, the multi-channel speech separation model still has greater advantages. Among them, the deep neural network (DNN) based beamformers (also known as neural beamformers) have achieved significant improvements in separation quality. The current neural beamformers can’t jointly optimize beamforming layers and DNN layers when using the prior knowledge of the existing beamforming algorithms, which may make the model unable to obtain the optimal separation performance. In order to solve this problem, this paper employs a set of beamformers that uniformly sample the space as a learning module in the neural network, and the initial values of their coefficients are determined by the existing maximum directivity factor (DF) beamformer. Furthermore, to obtain beam representations of source signals when their directions are unknown, a cross-attention mechanism is introduced. The experimental results show that in the separation task with reverberation, the proposed method has better performance than the current state-of-the-art temporal neural beamformer filter-and-sum network (FasNet) and several mainstream multi-channel speech separation approaches in terms of scale-invariant signal-to-noise ratio (SI-SNR), perceptual evaluation of speech quality (PESQ) and short-time objective intelligibility measure (STOI).

A Study of Learning Based Beamforming Methods for Speech Recognition

Deep Long Short-Term Memory Adaptive Beamforming Networks For Multichannel Robust Speech Recognition

Robust Beamforming for Speech Recognition Using DNN-Based Time-Frequency Masks Estimation.

Robust speech recognition using beamforming with adaptive microphone gains and multichannel noise reduction

An Implementaion of the CNN-Based MVDR Beamforming For Speech Enhancement.

Deep Learning Based Speech Beamforming

Beamforming-based Speech Enhancement Based on Optimal Ratio Mask

Robust mmWave Beamforming by Self-Supervised Hybrid Deep Learning

Masking-based Neural Beamformer for Multichannel Speech Enhancement

Neural Network Adaptive Beamforming for Robust Multichannel Speech Recognition

Robust Mask Estimation by Integrating Neural Network-Based and Clustering-Based Approaches for Adaptive Acoustic Beamforming.

Generalized RNN Beamformer for Target Speech Separation.

A New Neural Beamformer for Multi-channel Speech Separation

A unified multichannel far-field speech recognition system: combining neural beamforming with attention based end-to-end model

A Corpus-Based Evaluation of Beamforming Techniques and Phase-Based Frequency Masking

Attention-Based Beamformer For Multi-Channel Speech Enhancement

Speech Enhancement with Robust Beamforming for Spatially Overlapped and Distributed Sources.

Continual Learning-Based Fast Beamforming Adaptation in Downlink MISO Systems

Benefits of triple acoustic beamforming during speech-on-speech masking and sound localization for bilateral cochlear-implant users

Locate and Beamform: Two-dimensional Locating All-neural Beamformer for Multi-channel Speech Separation

A Deep Learning Approach to Location- and Orientation-aided 3D Beam Selection for mmWave Communications