Abstract:The hybrid deep neural network (DNN) and hidden Markov model (HMM) has recently achieved dramatic performance gains in automatic speech recognition (ASR). The DNN-based acoustic model is very powerful but its learning process is extremely time-consuming. In this paper, we propose a novel DNN-based acoustic modeling framework for speech recognition, where the posterior probabilities of HMM states are computed from multiple DNNs (mDNN), instead of a single large DNN, for the purpose of parallel training towards faster turnaround. In the proposed mDNN method all tied HMM states are first grouped into several disjoint clusters based on data-driven methods. Next, several hierarchically structured DNNs are trained separately in parallel for these clusters using multiple computing units (e.g. GPUs). In decoding, the posterior probabilities of HMM states can be calculated by combining outputs from multiple DNNs. In this work, we have shown that the training procedure of the mDNN under popular criteria, including both frame-level cross-entropy and sequence-level discriminative training, can be parallelized efficiently to yield significant speedup. The training speedup is mainly attributed to the fact that multiple DNNs are parallelized over multiple GPUs and each DNN is smaller in size and trained by only a subset of training data. We have evaluated the proposed mDNN method on a 64-hour Mandarin transcription task and the 320-hour Switchboard task. Compared to the conventional DNN, a 4-cluster mDNN model with similar size can yield comparable recognition performance in Switchboard (only about 2% performance degradation) with a greater than 7 times speed improvement in CE training and a 2.9 times improvement in sequence training, when 4 GPUs are used.

An efficient viterbi algorithm on DBNs

Bipolar Population Threshold Encoding for Audio Recognition with Deep Spiking Neural Networks

Improved Bottleneck Feature Using Hierarchical Deep Belief Networks for Keyword Spotting in Continues Speech

A tutorial of techniques for improving standard Hidden Markov Model algorithms

A k-climax neighbors policy based viterbi decoding for large vocabulary mandarin speech recognition

On Training Recurrent Networks with Truncated Backpropagation Through Time in Speech Recognition

Dynamic Bayesian network approach to speaker identification

Construction of a compact dynamic decoder network for large vocabulary continuous speech recognition

A Cluster-Based Multiple Deep Neural Networks Method for Large Vocabulary Continuous Speech Recognition

Fast learning algorithms for time-delay neural networks phoneme recognition

Improving deep neural networks for LVCSR using dropout and shrinking structure

Efficient One-Pass Decoding with Nnlm for Speech Recognition

Effective Phoneme Decoding With Hyperbolic Neural Networks for High-Performance Speech BCIs

Inference skipping for more efficient real-time speech enhancement with parallel RNNs

A Neural Network Approach To Viterbi Algorithm Based On Mfa

Recurrent Dirichlet Belief Networks for Interpretable Dynamic Relational Data Modelling

The Appropriate Hidden Layers of Deep Belief Networks for Speech Recognition

State-Clustering Based Multiple Deep Neural Networks Modeling Approach for Speech Recognition

A Fast Learning Algorithm for Time-Delay Neural Networks

ViterbiNet: A Deep Learning Based Viterbi Algorithm for Symbol Detection

Bayesian Learning of LF-MMI Trained Time Delay Neural Networks for Speech Recognition