Abstract:The application of data-intensive automatic speech recognition (ASR) technologies to dysarthric and elderly adult speech is confronted by their mismatch against healthy and nonaged voices, data scarcity and large speaker-level variability. To this end, this paper proposes two novel data-efficient methods to learn homogeneous dysarthric and elderly speaker-level features for rapid, on-the-fly test-time adaptation of DNN/TDNN and Conformer ASR models. These include: 1) speaker-level variance-regularized spectral basis embedding (VR-SBE) features that exploit a special regularization term to enforce homogeneity of speaker features in adaptation; and 2) feature-based learning hidden unit contributions (f-LHUC) transforms that are conditioned on VR-SBE features. Experiments are conducted on four tasks across two languages: the English UASpeech and TORGO dysarthric speech datasets, the English DementiaBank Pitt and Cantonese JCCOCC MoCA elderly speech corpora. The proposed on-the-fly speaker adaptation techniques consistently outperform baseline iVector and xVector adaptation by statistically significant word or character error rate reductions up to 5.32% absolute (18.57% relative) and batch-mode LHUC speaker adaptation by 2.24% absolute (9.20% relative), while operating with real-time factors speeding up to 33.6 times against xVectors during adaptation. The efficacy of the proposed adaptation techniques is demonstrated in a comparison against current ASR technologies including SSL pre-trained systems on UASpeech, where our best system produces a state-of-the-art WER of 23.33%. Analyses show VR-SBE features and f-LHUC transforms are insensitive to speaker-level data quantity in testtime adaptation. T-SNE visualization reveals they have stronger speaker-level homogeneity than baseline iVectors, xVectors and batch-mode LHUC transforms.

Multigrained Model Adaptation With Map And Reference Speaker Weighting For Text Independent Speaker Verification

Speaker Adaptation with MAP Estimation and Weighted Neighbor Regression

Speaker adaptation based on combination of MAP estimation and weighted neighbor regression

Agmma: A Novel Incremental Adaptation Method And Its Application To Speaker Recognition

MAP-based Speaker Adaptation in Speech Synthesis

Speaker Adaptation of Hybrid NN/HMM Model for Speech Recognition Based on Singular Value Decomposition

Sequential UBM Adaptation for Speaker Verification

Speaker-Smoothed kNN Speaker Adaptation for End-to-End ASR

A Novel speaker Adaptation Method based on Map and NNLR

Speaker Normalization Training and Adaptation for Speech Recognition

Unsupervised Speaker Adaptation Of Deep Neural Network Based On The Combination Of Speaker Codes And Singular Value Decomposition For Speech Recognition

Phoneme Dependent Speaker Embedding And Model Factorization For Multi-Speaker Speech Synthesis And Adaptation

A New Subspace Based Speaker Adaptation Method

Speaker Adaptation for Telephony Data Using Speaker Clustering

An New Approach for Incremental Speaker Adaptation

On-the-Fly Feature Based Rapid Speaker Adaptation for Dysarthric and Elderly Speech Recognition

Homogeneous Speaker Features for On-the-Fly Dysarthric and Elderly Speaker Adaptation

Speaker adaptation using maximum likelihood model interpolation

Batch Normalization based Unsupervised Speaker Adaptation for Acoustic Models

Multi-source Domain Adaptation for Text-independent Forensic Speaker Recognition

Efficient Black-Box Speaker Verification Model Adaptation with Reprogramming and Backend Learning