Abstract:The hybrid deep neural network (DNN) and hidden Markov model (HMM) has recently achieved dramatic performance gains in automatic speech recognition (ASR). The DNN-based acoustic model is very powerful but its learning process is extremely time-consuming. In this paper, we propose a novel DNN-based acoustic modeling framework for speech recognition, where the posterior probabilities of HMM states are computed from multiple DNNs (mDNN), instead of a single large DNN, for the purpose of parallel training towards faster turnaround. In the proposed mDNN method all tied HMM states are first grouped into several disjoint clusters based on data-driven methods. Next, several hierarchically structured DNNs are trained separately in parallel for these clusters using multiple computing units (e.g. GPUs). In decoding, the posterior probabilities of HMM states can be calculated by combining outputs from multiple DNNs. In this work, we have shown that the training procedure of the mDNN under popular criteria, including both frame-level cross-entropy and sequence-level discriminative training, can be parallelized efficiently to yield significant speedup. The training speedup is mainly attributed to the fact that multiple DNNs are parallelized over multiple GPUs and each DNN is smaller in size and trained by only a subset of training data. We have evaluated the proposed mDNN method on a 64-hour Mandarin transcription task and the 320-hour Switchboard task. Compared to the conventional DNN, a 4-cluster mDNN model with similar size can yield comparable recognition performance in Switchboard (only about 2% performance degradation) with a greater than 7 times speed improvement in CE training and a 2.9 times improvement in sequence training, when 4 GPUs are used.

Submodular Rank Aggregation on Score-based Permutations for Distributed Automatic Speech Recognition

Unsupervised Submodular Rank Aggregation on Score-based Permutations.

Robust Submodular Data Partitioning for Distributed Speech Recognition

Distributed Submodular Maximization for Large Vocabulary Continuous Speech Recognition

Distributed Training of Deep Neural Network Acoustic Models for Automatic Speech Recognition: A comparison of current training strategies

SpeechColab Leaderboard: An Open-Source Platform for Automatic Speech Recognition Evaluation

State-Clustering Based Multiple Deep Neural Networks Modeling Approach for Speech Recognition

Integrating Lattice-Free MMI into End-to-End Speech Recognition

Full-Rank No More: Low-Rank Weight Training for Modern Speech Recognition Models

Confidence-features and Confidence-Scores for ASR Applications in Arbitration and DNN Speaker Adaptation.

Numerical methods for distributed stochastic compositional optimization problems with aggregative structure

Automatic Speech Disentanglement for Voice Conversion using Rank Module and Speech Augmentation

A hybrid discriminant fuzzy DNN with enhanced modularity bat algorithm for speech recognition

Adaptive Consensus Gradients Aggregation for Scaled Distributed Training

Rank Aggregation Using Scoring Rules

Revisiting the Entropy Semiring for Neural Speech Recognition

High Fidelity Speech Enhancement with Band-split RNN

Advanced Recurrent Network-Based Hybrid Acoustic Models for Low Resource Speech Recognition

Unifying Speech Enhancement and Separation with Gradient Modulation for End-to-End Noise-Robust Speech Separation

Houston we have a Divergence: A Subgroup Performance Analysis of ASR Models

Exploring Speech Recognition, Translation, and Understanding with Discrete Speech Units: A Comparative Study