Abstract:Speech bandwidth extension (BWE) refers to widening the frequency bandwidth range of speech signals, enhancing the speech quality towards brighter and fuller. This paper proposes a generative adversarial network (GAN) based BWE model with parallel prediction of Amplitude and Phase spectra, named AP-BWE, which achieves both high-quality and efficient wideband speech waveform generation. The proposed AP-BWE generator is entirely based on convolutional neural networks (CNNs). It features a dual-stream architecture with mutual interaction, where the amplitude stream and the phase stream communicate with each other and respectively extend the high-frequency components from the input narrowband amplitude and phase spectra. To improve the naturalness of the extended speech signals, we employ a multi-period discriminator at the waveform level and design a pair of multi-resolution amplitude and phase discriminators at the spectral level, respectively. Experimental results demonstrate that our proposed AP-BWE achieves state-of-the-art performance in terms of speech quality for BWE tasks targeting sampling rates of both 16 kHz and 48 kHz. In terms of generation efficiency, due to the all-convolutional architecture and all-frame-level operations, the proposed AP-BWE can generate 48 kHz waveform samples 292.3 times faster than real-time on a single RTX 4090 GPU and 18.1 times faster than real-time on a single CPU. Notably, to our knowledge, AP-BWE is the first to achieve the direct extension of the high-frequency phase spectrum, which is beneficial for improving the effectiveness of existing BWE methods.

Waveform Modeling and Generation Using Hierarchical Recurrent Neural Networks for Speech Bandwidth Extension

Waveform Modeling Using Stacked Dilated Convolutional Neural Networks for Speech Bandwidth Extension.

Speech Bandwidth Extension Using Bottleneck Features and Deep Recurrent Neural Networks.

Restoring High Frequency Spectral Envelopes Using Neural Networks For Speech Bandwidth Extension

Recurrent neural network for spectral mapping in speech bandwidth extension

A Novel Unified Framework for Speech Enhancement and Bandwidth Extension Based on Jointly Trained Neural Networks

Speech Bandwidth Extension Based on Restricted Boltzmann Machines

Towards High-Quality and Efficient Speech Bandwidth Extension with Parallel Amplitude and Phase Prediction

Hierarchical RNNs for Waveform-Level Speech Synthesis

Audio Bandwidth Extension Using Ensemble of Recurrent Neural Networks

DSP-informed bandwidth extension using locally-conditioned excitation and linear time-varying filter subnetworks

Speech bandwidth extension supported by temporal information

Audio bandwidth extension based on RBF neural network

Speech Bandwidth Extension Using Recurrent Temporal Restricted Boltzmann Machines

Using conditional restricted Boltzmann machines for spectral envelope modeling in speech bandwidth extension.

An Experimental Study on Joint Modeling of Mixed-Bandwidth Data Via Deep Neural Networks for Robust Speech Recognition.

A Novel Research to Artificial Bandwidth Extension Based on Deep BLSTM Recurrent Neural Networks and Exemplar-Based Sparse Representation.

Speech Bandwidth Expansion Based on Deep Neural Networks

Multi-Stage Speech Bandwidth Extension with Flexible Sampling Rate Control

Nonlinear bandwidth extension of audio signals based on hidden Markov model

Blind Bandwidth Extension Of Audio Signals Based On Non-Linear Prediction And Hidden Markov Model