Abstract:The hybrid deep neural network (DNN) and hidden Markov model (HMM) has recently achieved dramatic performance gains in automatic speech recognition (ASR). The DNN-based acoustic model is very powerful but its learning process is extremely time-consuming. In this paper, we propose a novel DNN-based acoustic modeling framework for speech recognition, where the posterior probabilities of HMM states are computed from multiple DNNs (mDNN), instead of a single large DNN, for the purpose of parallel training towards faster turnaround. In the proposed mDNN method all tied HMM states are first grouped into several disjoint clusters based on data-driven methods. Next, several hierarchically structured DNNs are trained separately in parallel for these clusters using multiple computing units (e.g. GPUs). In decoding, the posterior probabilities of HMM states can be calculated by combining outputs from multiple DNNs. In this work, we have shown that the training procedure of the mDNN under popular criteria, including both frame-level cross-entropy and sequence-level discriminative training, can be parallelized efficiently to yield significant speedup. The training speedup is mainly attributed to the fact that multiple DNNs are parallelized over multiple GPUs and each DNN is smaller in size and trained by only a subset of training data. We have evaluated the proposed mDNN method on a 64-hour Mandarin transcription task and the 320-hour Switchboard task. Compared to the conventional DNN, a 4-cluster mDNN model with similar size can yield comparable recognition performance in Switchboard (only about 2% performance degradation) with a greater than 7 times speed improvement in CE training and a 2.9 times improvement in sequence training, when 4 GPUs are used.

Audio-Visual Tibetan Speech Recognition Based On A Deep Dynamic Bayesian Network For Natural Human Robot Interaction Regular Paper

Dynamic bayesian networks for audio-visual speaker recognition

Speech Recognition Based on Deep Neural Networks on Tibetan Corpus

Automatic Speaker Recognition Using Dynamic Bayesian Network.

Dbn Based Multi-Stream Models For Speech

Deep Neural Network based Uyghur Large Vocabulary Continuous Speech Recognition

Multi-task Joint-Learning for Robust Voice Activity Detection

An Audio-Visual Speech Recognition Framework Based on Articulatory Features.

Audio-visual multi-channel speech separation, dereverberation and recognition

Robust end-to-end deep audiovisual speech recognition

Bimodal speaker identification using dynamic bayesian network

Multi-task Joint-Learning of Deep Neural Networks for Robust Speech Recognition

Simultaneous facial activity tracking and recognition

Robust Audio-visual Speech Recognition Using Bimodal Dfsmn with Multi-condition Training and Dropout Regularization.

Audio Visual Speech Recognition with Multimodal Recurrent Neural Networks

State-Clustering Based Multiple Deep Neural Networks Modeling Approach for Speech Recognition

Dynamic TF-TDNN: Dynamic Time Delay Neural Network Based on Temporal-Frequency Attention for Dialect Recognition

End-to-end Audiovisual Speech Activity Detection with Bimodal Recurrent Neural Models

Look, Listen and Learn - A Multimodal LSTM for Speaker Identification

Mining Audio/Visual Database For Speech Driven Face Animation

Collaborative Joint Training With Multitask Recurrent Model for Speech and Speaker Recognition.