Abstract:A key task for speech recognition systems is to reduce the mismatch between training and evaluation data that is often attributable to speaker differences. Speaker adaptation techniques play a vital role to reduce the mismatch. Model-based speaker adaptation approaches often require sufficient amounts of target speaker data to ensure robustness. When the amount of speaker level data is limited, speaker adaptation is prone to overfitting and poor generalization. To address the issue, this paper proposes a full Bayesian learning based DNN speaker adaptation framework to model speaker-dependent (SD) parameter uncertainty given limited speaker specific adaptation data. This framework is investigated in three forms of model based DNN adaptation techniques: Bayesian learning of hidden unit contributions (BLHUC), Bayesian parameterized activation functions (BPAct), and Bayesian hidden unit bias vectors (BHUB). In the three methods, deterministic SD parameters are replaced by latent variable posterior distributions for each speaker, whose parameters are efficiently estimated using a variational inference based approach. Experiments conducted on 300-hour speed perturbed Switchboard corpus trained LF-MMI TDNN/CNN-TDNN systems suggest the proposed Bayesian adaptation approaches consistently outperform the deterministic adaptation on the NIST Hub5'00 and RT03 evaluation sets. When using only the first five utterances from each speaker as adaptation data, significant word error rate reductions up to 1.4% absolute (7.2% relative) were obtained on the CallHome subset. The efficacy of the proposed Bayesian adaptation techniques is further demonstrated in a comparison against the state-of-the-art performance obtained on the same task using the most recent systems reported in the literature.

Probabilistic Latent Speaker Training for Large Vocabulary Speech Recognition

Probabilistic Latent Speaker Analysis for Large Vocabulary Speech Recognition

Probabilistic Speaker-Class Based Acoustic Modeling for Large Vocabulary Continuous Speech Recognition

AudioVSR: Enhancing Video Speech Recognition with Audio Data

Latent Correlation Analysis of HMM Parameters for Speech Recognition

A New Subspace Based Speaker Adaptation Method

Speaker-aware Training of LSTM-RNNS for Acoustic Modelling

Factorised Speaker-environment Adaptive Training of Conformer Speech Recognition Systems

Speaker Adaptation of Hybrid NN/HMM Model for Speech Recognition Based on Singular Value Decomposition

Hidden Markov Acoustic Modeling with Bootstrap and Restructuring for Low-Resourced Languages

Unsupervised Speaker Adaptation of Blstm-Rnn for Lvcsr Based on Speaker Code

Model Adaptation Using the Projection to Latent Structure Algorithm

Improving Online Incremental Speaker Adaptation with Eigen Feature Space MLLR.

A Probabilistic Principal Component Analysis Based Hidden Markov Model For Audio-Visual Speech Recognition

Modeling Speaker Variability Using Long Short-Term Memory Networks For Speech Recognition

Direct adaptation of hybrid DNN/HMM model for fast speaker adaptation in LVCSR based on speaker code

Speaker adaptive bottleneck features extraction for LVCSR based on discriminative learning of speaker codes

Bayesian Learning for Deep Neural Network Adaptation

Bayesian Adaptive Inference and Adaptive Training.

Discriminative Transformation for Sufficient Adaptation in Text-Independent Speaker Verification

Phoneme Dependent Speaker Embedding And Model Factorization For Multi-Speaker Speech Synthesis And Adaptation