Abstract:The hybrid deep neural network (DNN) and hidden Markov model (HMM) has recently achieved dramatic performance gains in automatic speech recognition (ASR). The DNN-based acoustic model is very powerful but its learning process is extremely time-consuming. In this paper, we propose a novel DNN-based acoustic modeling framework for speech recognition, where the posterior probabilities of HMM states are computed from multiple DNNs (mDNN), instead of a single large DNN, for the purpose of parallel training towards faster turnaround. In the proposed mDNN method all tied HMM states are first grouped into several disjoint clusters based on data-driven methods. Next, several hierarchically structured DNNs are trained separately in parallel for these clusters using multiple computing units (e.g. GPUs). In decoding, the posterior probabilities of HMM states can be calculated by combining outputs from multiple DNNs. In this work, we have shown that the training procedure of the mDNN under popular criteria, including both frame-level cross-entropy and sequence-level discriminative training, can be parallelized efficiently to yield significant speedup. The training speedup is mainly attributed to the fact that multiple DNNs are parallelized over multiple GPUs and each DNN is smaller in size and trained by only a subset of training data. We have evaluated the proposed mDNN method on a 64-hour Mandarin transcription task and the 320-hour Switchboard task. Compared to the conventional DNN, a 4-cluster mDNN model with similar size can yield comparable recognition performance in Switchboard (only about 2% performance degradation) with a greater than 7 times speed improvement in CE training and a 2.9 times improvement in sequence training, when 4 GPUs are used.

Cluster-Based Senone Selection for the Efficient Calculation of Deep Neural Network Acoustic Models

Improving the Decoding Efficiency of Deep Neural Network Acoustic Models by Cluster-Based Senone Selection.

State-Clustering Based Multiple Deep Neural Networks Modeling Approach for Speech Recognition

An Investigation of High-Resolution Modeling Units of Deep Neural Networks for Acoustic Scene Classification

Speaker Cluster-Based Speaker Adaptive Training for Deep Neural Network Acoustic Modeling

Acceleration Strategies for Speech Recognition Based on Deep Neural Networks

A Cluster-Based Multiple Deep Neural Networks Method for Large Vocabulary Continuous Speech Recognition

Phonotactic language recognition based on DNN-HMM acoustic model

Fast Adaptation on Deepmixture Generative Network Based Acoustic Modeling

Bipolar Population Threshold Encoding for Audio Recognition with Deep Spiking Neural Networks

Deep Neural Network Based Noised Asian Speech Enhancement and Its Implementation on a Hearing Aid App.

An Acoustic Model for English Speech Recognition Based on Deep Learning

Empirically Combining Unnormalized NNLM and Back-off N -Gram for Fast N -Best Rescoring in Speech Recognition

Unit Selection Speech Synthesis Using Frame-Sized Speech Segments and Neural Network Based Acoustic Models

DNN-based unit selection using frame-sized speech segments

Prediction of speech intelligibility with DNN-based performance measures

LID-senone Extraction Via Deep Neural Networks for End-to-End Language Identification

Sequential Cluster Method and Its Application on Neural Network Based Speech Recognition

Deep neural networks based speaker modeling at different levels of phonetic granularity

Cluster Adaptive Training for Deep Neural Network