Abstract:Automatic speech recognition (ASR) systems are often built using scene related speech data due to large variations of transmission channels and sampling rates in different scenarios. In this study, we propose a general framework that establishes a unified model for diversified speech data with different sampling rates and channels. The framework is a joint optimization of deep neural network (DNN)-based bandwidth expansion and acoustic modeling to exploit a large amount of diversified training data. First, we design two novel DNN architectures to map the acoustic features from narrowband to wideband speech through direct mapping and progressive mapping. The learning targets of the direct mapping DNN (DNN-DM) are the acoustic features extracted from speech with the largest bandwidth, while the acoustic features from speech with all the other bandwidths are used as input. A progressive stacking network (PSN) gradually maps the features from the low sampling rates to the highest sampling rate through the design of intermediate target layers via multitask training. Then, in addition to these bandwidth expansion networks, we investigate several joint training strategies for DNN-based acoustic models. Our experiments conducted on three diversified large-scale Mandarin speech datasets with different recording channels and sampling rates (6,8, and 16 kHz) show that the proposed unified model using PSN for bandwidth expansion not only is a more flexible and compact design than conventional multiple acoustic models with each bandwidth for a specific sampling rate, but also yields consistent and significant improvements over bandwidth-dependent models with an average relative word error rate reduction of 6.2%, indicating that the proposed model can fully utilize the diversified cross-channel speech data with multiple bandwidths. Moreover, the proposed methods are verified to be robust on different realistic scenes and can be effectively extended to a long short-term memory framework.

Waveform Modeling Using Stacked Dilated Convolutional Neural Networks for Speech Bandwidth Extension.

Waveform Modeling and Generation Using Hierarchical Recurrent Neural Networks for Speech Bandwidth Extension

Speech Bandwidth Extension Using Bottleneck Features and Deep Recurrent Neural Networks.

Restoring High Frequency Spectral Envelopes Using Neural Networks For Speech Bandwidth Extension

A Novel Unified Framework for Speech Enhancement and Bandwidth Extension Based on Jointly Trained Neural Networks

Towards High-Quality and Efficient Speech Bandwidth Extension with Parallel Amplitude and Phase Prediction

DSP-informed bandwidth extension using locally-conditioned excitation and linear time-varying filter subnetworks

Multi-Stage Speech Bandwidth Extension with Flexible Sampling Rate Control

Vector Quantized Diffusion Model Based Speech Bandwidth Extension

An Experimental Study on Joint Modeling of Mixed-Bandwidth Data Via Deep Neural Networks for Robust Speech Recognition.

BAE-Net: A Low complexity and high fidelity Bandwidth-Adaptive neural network for speech super-resolution

A Novel Research to Artificial Bandwidth Extension Based on Deep BLSTM Recurrent Neural Networks and Exemplar-Based Sparse Representation.

A Novel Method of Artificial Bandwidth Extension Using Deep Architecture.

Speech Bandwidth Expansion Via High Fidelity Generative Adversarial Networks

Stochastic WaveNet: A Generative Latent Variable Model for Sequential Data

Time-Domain Neural Network Approach for Speech Bandwidth Extension.

Mixed-Bandwidth Cross-Channel Speech Recognition Via Joint Optimization of DNN-Based Bandwidth Expansion and Acoustic Modeling.

Neural Speech Enhancement with Very Low Algorithmic Latency and Complexity via Integrated Full- and Sub-Band Modeling

Hybrid Transform Based Speech Band Width Enhancement Using Data Hiding

Quasi-Periodic WaveNet: An Autoregressive Raw Waveform Generative Model With Pitch-Dependent Dilated Convolution Neural Network

A two-stage full-band speech enhancement model with effective spectral compression mapping