Abstract:The hybrid deep neural network (DNN) and hidden Markov model (HMM) has recently achieved dramatic performance gains in automatic speech recognition (ASR). The DNN-based acoustic model is very powerful but its learning process is extremely time-consuming. In this paper, we propose a novel DNN-based acoustic modeling framework for speech recognition, where the posterior probabilities of HMM states are computed from multiple DNNs (mDNN), instead of a single large DNN, for the purpose of parallel training towards faster turnaround. In the proposed mDNN method all tied HMM states are first grouped into several disjoint clusters based on data-driven methods. Next, several hierarchically structured DNNs are trained separately in parallel for these clusters using multiple computing units (e.g. GPUs). In decoding, the posterior probabilities of HMM states can be calculated by combining outputs from multiple DNNs. In this work, we have shown that the training procedure of the mDNN under popular criteria, including both frame-level cross-entropy and sequence-level discriminative training, can be parallelized efficiently to yield significant speedup. The training speedup is mainly attributed to the fact that multiple DNNs are parallelized over multiple GPUs and each DNN is smaller in size and trained by only a subset of training data. We have evaluated the proposed mDNN method on a 64-hour Mandarin transcription task and the 320-hour Switchboard task. Compared to the conventional DNN, a 4-cluster mDNN model with similar size can yield comparable recognition performance in Switchboard (only about 2% performance degradation) with a greater than 7 times speed improvement in CE training and a 2.9 times improvement in sequence training, when 4 GPUs are used.

Fast learning algorithms for time-delay neural networks phoneme recognition

A Fast Learning Algorithm for Time-Delay Neural Networks

A Fast Learning Algorithm Of Neural Network For The Training And Recognition Of The Phonemes

Deep Time Delay Neural Network for Speech Enhancement with Full Data Learning

Accelerated Information Processing Based on Deep Photonic Time-Delay Reservoir Computing

Fast Learning Algorithms for Feedforward Neural Networks

Improved TDNNs Using Deep Kernels and Frequency Dependent Grid-RNNs.

New Neural Network Architecture with Application in Mandarin Digit Speech Recognition

Learning Dynamic Features with Neural Networks for Phoneme Recognition

EfficientTDNN: Efficient Architecture Search for Speaker Recognition

Deep Recurrent Convolutional Neural Network: Improving Performance For Speech Recognition

Acceleration Strategies for Speech Recognition Based on Deep Neural Networks

Densely Connected Time Delay Neural Network for Speaker Verification.

Investigation of Tandem Deep Belief Network Approach for Phoneme Recognition

Voice Activity Detection Based on Time-Delay Neural Networks

On the Application and Compression of Deep Time Delay Neural Network for Embedded Statistical Parametric Speech Synthesis

A Dynamic Neural Network for Syllable Recognition.

Improved Phonotactic Language Recognition Based on RNN Feature Reconstruction

Multi-Loss Convolutional Network with Time-Frequency Attention for Speech Enhancement

State-Clustering Based Multiple Deep Neural Networks Modeling Approach for Speech Recognition

Time-Frequency Cepstral Features and Combining Discriminative Training for Phonotactic Language Recognition