Abstract:Fast adaptation of deep neural networks (DNN) is an important research topic in deep learning. In this paper, we have proposed a general adaptation scheme for DNN based on discriminant condition codes, which are directly fed to various layers of a pre-trained DNN through a new set of connection weights. Moreover, we present several training methods to learn connection weights from training data as well as the corresponding adaptation methods to learn new condition code from adaptation data for each new test condition. In this work, the fast adaptation scheme is applied to supervised speaker adaptation in speech recognition based on either frame-level cross-entropy or sequence-level maximum mutual information training criterion. We have proposed three different ways to apply this adaptation scheme based on the so-called speaker codes: i) Nonlinear feature normalization in feature space; ii) Direct model adaptation of DNN based on speaker codes; iii) Joint speaker adaptive training with speaker codes. We have evaluated the proposed adaptation methods in two standard speech recognition tasks, namely TIMIT phone recognition and large vocabulary speech recognition in the Switchboard task. Experimental results have shown that all three methods are quite effective to adapt large DNN models using only a small amount of adaptation data. For example, the Switchboard results have shown that the proposed speaker-code-based adaptation methods may achieve up to 8-10% relative error reduction using only a few dozens of adaptation utterances per speaker. Finally, we have achieved very good performance in Switchboard (12.1% in WER) after speaker adaptation using sequence training criterion, which is very close to the best performance reported in this task (“Deep convolutional neural networks for LVCSR,” T. N. Sainath , Proc. IEEE Acoust., Speech, Signal Process., 2013).

Dynamic Speaker Selected Training for Rapid Speaker Adaptation

Agmma: A Novel Incremental Adaptation Method And Its Application To Speaker Recognition

Real-time Speaker Recognition System for PDA

Discriminative Dynamic Gaussian Mixture Selection with Enhanced Robustness and Performance for Multi-Accent Speech Recognition

Speech Recognition Using Speaker Adaptation by System Parameter Transformation.

Dynamic Speaker Representations Adjustment and Decoder Factorization for Speaker Adaptation in End-to-End Speech Synthesis

An New Approach for Incremental Speaker Adaptation

Factorised Speaker-environment Adaptive Training of Conformer Speech Recognition Systems

Codebook-Based Speaker Adaptation

Fast Adaptation of Deep Neural Network Based on Discriminant Codes for Speech Recognition

Rapid discriminative acoustic model based on eigenspace mapping for fast speaker adaptation

Acceleration Strategies for Speech Recognition Based on Deep Neural Networks

Speaker-Smoothed kNN Speaker Adaptation for End-to-End ASR

Text-independent Speaker Recognition Based on Self-adaptation Compensation Transformation

Reliable Accent-Specific Unit Generation With Discriminative Dynamic Gaussian Mixture Selection for Multi-Accent Chinese Speech Recognition

Rapid Speaker Adaptation Using Multi-Stream Structural Maximum Likelihood Eigenspace Mapping

Bayesian Learning for Deep Neural Network Adaptation

Speaker Adaptation and Adaptive Training for Jointly Optimised Tandem Systems.

Speaker Adaptation Using Spectro-Temporal Deep Features for Dysarthric and Elderly Speech Recognition

Speaker Adaptation of Hybrid NN/HMM Model for Speech Recognition Based on Singular Value Decomposition

A New Subspace Based Speaker Adaptation Method