Abstract:Automatic speech recognition (ASR) systems are often built using scene related speech data due to large variations of transmission channels and sampling rates in different scenarios. In this study, we propose a general framework that establishes a unified model for diversified speech data with different sampling rates and channels. The framework is a joint optimization of deep neural network (DNN)-based bandwidth expansion and acoustic modeling to exploit a large amount of diversified training data. First, we design two novel DNN architectures to map the acoustic features from narrowband to wideband speech through direct mapping and progressive mapping. The learning targets of the direct mapping DNN (DNN-DM) are the acoustic features extracted from speech with the largest bandwidth, while the acoustic features from speech with all the other bandwidths are used as input. A progressive stacking network (PSN) gradually maps the features from the low sampling rates to the highest sampling rate through the design of intermediate target layers via multitask training. Then, in addition to these bandwidth expansion networks, we investigate several joint training strategies for DNN-based acoustic models. Our experiments conducted on three diversified large-scale Mandarin speech datasets with different recording channels and sampling rates (6,8, and 16 kHz) show that the proposed unified model using PSN for bandwidth expansion not only is a more flexible and compact design than conventional multiple acoustic models with each bandwidth for a specific sampling rate, but also yields consistent and significant improvements over bandwidth-dependent models with an average relative word error rate reduction of 6.2%, indicating that the proposed model can fully utilize the diversified cross-channel speech data with multiple bandwidths. Moreover, the proposed methods are verified to be robust on different realistic scenes and can be effectively extended to a long short-term memory framework.

Model Synthesis for Band-Limited Speech Recognition.

Research on Bandwidth Mismatch Compensation in Speech Recognition

Synthesis of wideband speech from narrow‐band speech based on piecewise linear mapping

Speech Bandwidth Extension Method Using Speech Recognition And Speech Synthesis

Sparse-Based Auditory Model for Robust Speaker Recognition

Improving Speech Recognition with Augmented Synthesized Data and Conditional Model Training

Towards Transfer Learning for End-to-End Speech Synthesis from Deep Pre-Trained Language Models.

Coding distortion compensation of speaker identification based on model synthesis

Artificial Bandwidth Extension For Speech Signals Using Speech Recogniton

A Novel Unified Framework for Speech Enhancement and Bandwidth Extension Based on Jointly Trained Neural Networks

Mixed-Bandwidth Cross-Channel Speech Recognition Via Joint Optimization of DNN-Based Bandwidth Expansion and Acoustic Modeling.

High-Fidelity Speech Synthesis with Minimal Supervision: All Using Diffusion Models

Multi-source Based Acoustic Model for Speech Synthesis.

SynesLM: A Unified Approach for Audio-visual Speech Recognition and Translation via Language Model and Synthetic Data

Improving Recognition-Synthesis Based Any-to-one Voice Conversion with Cyclic Training

Phonetic Enhanced Language Modeling for Text-to-Speech Synthesis

Synthesized Stereo-Based Stochastic Mapping with Data Selection for Robust Speech Recognition.

Restricted Boltzmann Machine Based Spectrum Modeling and Unit Selection Speech Synthesis Method

Speech Recognition with Augmented Synthesized Speech

ZMM-TTS: Zero-shot Multilingual and Multispeaker Speech Synthesis Conditioned on Self-supervised Discrete Speech Representations

New synthesis method based on LMA vocal tract model