Abstract:Fast adaptation of deep neural networks (DNN) is an important research topic in deep learning. In this paper, we have proposed a general adaptation scheme for DNN based on discriminant condition codes, which are directly fed to various layers of a pre-trained DNN through a new set of connection weights. Moreover, we present several training methods to learn connection weights from training data as well as the corresponding adaptation methods to learn new condition code from adaptation data for each new test condition. In this work, the fast adaptation scheme is applied to supervised speaker adaptation in speech recognition based on either frame-level cross-entropy or sequence-level maximum mutual information training criterion. We have proposed three different ways to apply this adaptation scheme based on the so-called speaker codes: i) Nonlinear feature normalization in feature space; ii) Direct model adaptation of DNN based on speaker codes; iii) Joint speaker adaptive training with speaker codes. We have evaluated the proposed adaptation methods in two standard speech recognition tasks, namely TIMIT phone recognition and large vocabulary speech recognition in the Switchboard task. Experimental results have shown that all three methods are quite effective to adapt large DNN models using only a small amount of adaptation data. For example, the Switchboard results have shown that the proposed speaker-code-based adaptation methods may achieve up to 8-10% relative error reduction using only a few dozens of adaptation utterances per speaker. Finally, we have achieved very good performance in Switchboard (12.1% in WER) after speaker adaptation using sequence training criterion, which is very close to the best performance reported in this task (“Deep convolutional neural networks for LVCSR,” T. N. Sainath , Proc. IEEE Acoust., Speech, Signal Process., 2013).

Alternating update layers for DBN-DNN fast training method

Fast learning algorithms for time-delay neural networks phoneme recognition

A Fast Learning Algorithm for Time-Delay Neural Networks

Improving deep neural networks for LVCSR using dropout and shrinking structure

A Pipelined Pre-Training Algorithm For Dbns

Research on Acceleration Method of Speech Recognition Training.

Automatic Model Redundancy Reduction for Fast Back-Propagation for Deep Neural Networks in Speech Recognition

A Layer-Based Sparsification Method for Distributed DNN Training.

Stochastic data sweeping for fast DNN training

A Fast Learning Algorithm Of Neural Network For The Training And Recognition Of The Phonemes

Fast Adaptation of Deep Neural Network Based on Discriminant Codes for Speech Recognition

Reshaping deep neural network for fast decoding by node-pruning

Fast Training Algorithm for Deep Neural Network Using Multiple GPUs

Rapid Adaptation For Deep Neural Networks Through Multi-Task Learning

Fast Learning Algorithms for Feedforward Neural Networks

State-Clustering Based Multiple Deep Neural Networks Modeling Approach for Speech Recognition

Extended Low-Rank Plus Diagonal Adaptation for Deep and Recurrent Neural Networks.

A Novel Training Strategy Using Dynamic Data Generation for Deep Neural Network Based Speech Enhancement.

Inference skipping for more efficient real-time speech enhancement with parallel RNNs

Empirical Evaluation of Parallel Training Algorithms on Acoustic Modeling

Exponential Moving Average Model in Parallel Speech Recognition Training