Abstract:The hybrid deep neural network (DNN) and hidden Markov model (HMM) has recently achieved dramatic performance gains in automatic speech recognition (ASR). The DNN-based acoustic model is very powerful but its learning process is extremely time-consuming. In this paper, we propose a novel DNN-based acoustic modeling framework for speech recognition, where the posterior probabilities of HMM states are computed from multiple DNNs (mDNN), instead of a single large DNN, for the purpose of parallel training towards faster turnaround. In the proposed mDNN method all tied HMM states are first grouped into several disjoint clusters based on data-driven methods. Next, several hierarchically structured DNNs are trained separately in parallel for these clusters using multiple computing units (e.g. GPUs). In decoding, the posterior probabilities of HMM states can be calculated by combining outputs from multiple DNNs. In this work, we have shown that the training procedure of the mDNN under popular criteria, including both frame-level cross-entropy and sequence-level discriminative training, can be parallelized efficiently to yield significant speedup. The training speedup is mainly attributed to the fact that multiple DNNs are parallelized over multiple GPUs and each DNN is smaller in size and trained by only a subset of training data. We have evaluated the proposed mDNN method on a 64-hour Mandarin transcription task and the 320-hour Switchboard task. Compared to the conventional DNN, a 4-cluster mDNN model with similar size can yield comparable recognition performance in Switchboard (only about 2% performance degradation) with a greater than 7 times speed improvement in CE training and a 2.9 times improvement in sequence training, when 4 GPUs are used.

Learning and Modeling Unit Embeddings Using Deep Neural Networks for Unit-Selection-Based Mandarin Speech Synthesis.

Learning and Modeling Unit Embeddings Using Deep Neural Networks for Unit-Selection-Based Mandarin Speech Synthesis

Learning and Modeling Unit Embeddings for Improving HMM-based Unit Selection Speech Synthesis

Extracting Unit Embeddings Using Sequence-To-Sequence Acoustic Models for Unit Selection Speech Synthesis

DNN-based unit selection using frame-sized speech segments

Unit Selection Speech Synthesis Using Frame-Sized Speech Segments and Neural Network Based Acoustic Models

Unitnet: A Sequence-To-Sequence Acoustic Model For Concatenative Speech Synthesis

Mongolian acoustic modeling based on deep neural network

Statistical Acoustic Model Based Unit Selection Algorithm for Speech Synthesis

A comparative study on selecting acoustic modeling units in deep neural networks based large vocabulary Chinese speech recognition

Multi-Level Modeling Units for End-to-End Mandarin Speech Recognition

UnitNet-Based Hybrid Speech Synthesis

State-Clustering Based Multiple Deep Neural Networks Modeling Approach for Speech Recognition

Hierarchical Non-Uniform Unit Selection Based on Prosodic Structure

Deep Metric Learning For The Target Cost In Unit-Selection Speech Synthesizer

Deep neural networks based speaker modeling at different levels of phonetic granularity

An Acoustic Model for English Speech Recognition Based on Deep Learning

Statistical modeling of syllable-level F0 features for HMM-based unit selection speech synthesis

HMM-based Unit Selection Using F

Trainable Unit Selection Speech Synthesis under Statistical Framework