Abstract:The hybrid deep neural network (DNN) and hidden Markov model (HMM) has recently achieved dramatic performance gains in automatic speech recognition (ASR). The DNN-based acoustic model is very powerful but its learning process is extremely time-consuming. In this paper, we propose a novel DNN-based acoustic modeling framework for speech recognition, where the posterior probabilities of HMM states are computed from multiple DNNs (mDNN), instead of a single large DNN, for the purpose of parallel training towards faster turnaround. In the proposed mDNN method all tied HMM states are first grouped into several disjoint clusters based on data-driven methods. Next, several hierarchically structured DNNs are trained separately in parallel for these clusters using multiple computing units (e.g. GPUs). In decoding, the posterior probabilities of HMM states can be calculated by combining outputs from multiple DNNs. In this work, we have shown that the training procedure of the mDNN under popular criteria, including both frame-level cross-entropy and sequence-level discriminative training, can be parallelized efficiently to yield significant speedup. The training speedup is mainly attributed to the fact that multiple DNNs are parallelized over multiple GPUs and each DNN is smaller in size and trained by only a subset of training data. We have evaluated the proposed mDNN method on a 64-hour Mandarin transcription task and the 320-hour Switchboard task. Compared to the conventional DNN, a 4-cluster mDNN model with similar size can yield comparable recognition performance in Switchboard (only about 2% performance degradation) with a greater than 7 times speed improvement in CE training and a 2.9 times improvement in sequence training, when 4 GPUs are used.

Deep Neural Network-based Mixed Speech Recognition Technology for Chinese and English

DLD: An Optimized Chinese Speech Recognition Model Based on Deep Learning

A hybrid CTC+Attention model based on end-to-end framework for multilingual speech recognition

English speech recognition based on deep learning with multiple features

Deep Neural Network Based Chinese Dialect Classification

Research on Chinese Speech Emotion Recognition Based on Deep Neural Network and Acoustic Features

Deep Speech 2: End-to-End Speech Recognition in English and Mandarin

Performance Optimization of Speech Recognition System with Deep Neural Network Model

State-Clustering Based Multiple Deep Neural Networks Modeling Approach for Speech Recognition

Multi-Channel Automatic Speech Recognition Using Deep Complex Unet

Multilingual Approach to Joint Speech and Accent Recognition with DNN-HMM Framework

A Convenient and Extensible Offline Chinese Speech Recognition System Based on Convolutional CTC Networks

Towards Language-Universal Mandarin-English Speech Recognition

A Unified Speaker-Dependent Speech Separation and Enhancement System Based on Deep Neural Networks.

Control System and Speech Recognition of Exhibition Hall Digital Media Based on Computer Technology

An Acoustic Model for English Speech Recognition Based on Deep Learning

Integrated Method of Deep Learning and Large Language Model in Speech Recognition

Research on Speech Emotion Recognition Technology based on Deep and Shallow Neural Network

New Neural Network Architecture with Application in Mandarin Digit Speech Recognition

Target Speech Signal Enhancement Based on Deep Neural Networks