Abstract:The hybrid deep neural network (DNN) and hidden Markov model (HMM) has recently achieved dramatic performance gains in automatic speech recognition (ASR). The DNN-based acoustic model is very powerful but its learning process is extremely time-consuming. In this paper, we propose a novel DNN-based acoustic modeling framework for speech recognition, where the posterior probabilities of HMM states are computed from multiple DNNs (mDNN), instead of a single large DNN, for the purpose of parallel training towards faster turnaround. In the proposed mDNN method all tied HMM states are first grouped into several disjoint clusters based on data-driven methods. Next, several hierarchically structured DNNs are trained separately in parallel for these clusters using multiple computing units (e.g. GPUs). In decoding, the posterior probabilities of HMM states can be calculated by combining outputs from multiple DNNs. In this work, we have shown that the training procedure of the mDNN under popular criteria, including both frame-level cross-entropy and sequence-level discriminative training, can be parallelized efficiently to yield significant speedup. The training speedup is mainly attributed to the fact that multiple DNNs are parallelized over multiple GPUs and each DNN is smaller in size and trained by only a subset of training data. We have evaluated the proposed mDNN method on a 64-hour Mandarin transcription task and the 320-hour Switchboard task. Compared to the conventional DNN, a 4-cluster mDNN model with similar size can yield comparable recognition performance in Switchboard (only about 2% performance degradation) with a greater than 7 times speed improvement in CE training and a 2.9 times improvement in sequence training, when 4 GPUs are used.

On modeling context-dependent clustered states: Comparing HMM/GMM, hybrid HMM/ANN and KL-HMM approaches

Enhanced Bayesian Gaussian hidden Markov mixture clustering for improved knowledge discovery

A New Hybrid Hmm/Ann Model For Speech Recognition

An Acoustic Model Based on HMM/ANN

Canonical State Models for Automatic Speech Recognition

Determine the Number of States in Hidden Markov Models via Marginal Likelihood

Improving the Performance of the LSTM and HMM Model via Hybridization

Direct modelling of output context dependence in discriminative hidden Markov model

Exploring More Representative States of Hidden Markov Model in Optical Character Recognition: A Clustering-Based Model Pre-Training Approach

Sticky Hidden Markov Modeling of Comparative Genomic Hybridization

Discriminative Hidden Markov Modeling with Long State Dependence Using a Knn Ensemble

Adaptive Bi-Weighting Toward Automatic Initialization and Model Selection for HMM-Based Hybrid Meta-Clustering Ensembles

Clustering hidden Markov models with variational HEM

Key Technology Research for Speech Recognition

State-Clustering Based Multiple Deep Neural Networks Modeling Approach for Speech Recognition

Decision tree based state tying for speech recognition using DNN derived embeddings

Towards interpretability of Mixtures of Hidden Markov Models

An Experimental Comparison of Hybrid Algorithms for Bayesian Network Structure Learning

Mixed Location Scale Hidden Markov Model for the Analysis of Intensive Longitudinal Data

DBHC: Discrete Bayesian HMM Clustering

Gaussian Latent Dirichlet Allocation for Discrete Human State Discovery