Monaural Speech Enhancement Using Deep Multi-Branch Residual Network with 1-D Causal Dilated Convolutions

Zhang Qiquan,Nicolson Aaron,Wang Mingjiang

2019-01-01

Abstract: Deep learning has achieved substantial improvement on single-channel speech enhancement tasks. However, the performance of multi-layer perceptions (MLPs)-based methods is limited by its ability to capture the long-term effective history information. Long short-term memory (LSTM) model is limited by its high computational complexity and the exploding/vanish gradients problem. In this paper, we propose a novel convolution neural network (CNN) architecture, called multi-branch residual network (MB-ResNet), for monaural speech enhancement. The MB-ResNet exploits split-transform-aggregate design, which is expected to obtain strong representational power at a low computational complexity. The combination of causal dilated one dimensional CNN and residual connection significantly expands receptive fields, which allows for capturing extremely long-term temporal contextual information. Our extensive experimental investigation suggest that the MB-ResNets outperform the residual long short-term memory networks (ResLSTMs) and the CNN networks that employ residual or dense aggregations in terms of speech intelligibility and quality, while providing superior parameter efficiency. Furthermore, our experimental results demonstrate that our proposed MB-ResNet model is able to outperform multiple state-of-the-art deep learning-based speech enhancement methods in terms of five widely used objective metrics.