Abstract:Despite the great success of speaker recognition models based on deep neural networks, deploying a pre-trained model in real-world scenarios often leads to significant performance degradation due to the domain mismatch between training and testing conditions. Various adaptation methods have been developed to address this issue by modifying either the front-end embedding network or the back-end scoring model. However, existing methods typically rely on knowledge of the network, scoring model, or even the source data. In this study, we introduce a knowledge-free adaptation approach that only necessitates the unlabeled target data. Our core concept is based on the assumption that domain mismatch primarily stems from distributional distortion in the embedding space, such as shifting, rotation, and scaling while maintaining inter-speaker discrimination for data from unknown domains. Building on this assumption, we propose clustering LDA (C-LDA), a full-rank linear discriminant analysis (LDA) based on agglomerative hierarchical clustering (AHC) to compensate for this distortion. This approach does not need any human labels and does not rely on any knowledge of the model in the source domain, making it suitable for real-world applications. Theoretical analysis indicates that with cosine scoring, C-LDA is capable of eliminating distributional distortion related to global shift and within-speaker covariance rotation and scaling. Surprisingly, our experiments demonstrated that this simple approach can outperform more complex methods that require full or partial knowledge, including front-end approaches such as fine-tuning and distribution alignment, and back-end approaches such as unsupervised probabilistic linear discriminant analysis (PLDA) adaptation. Additional experiments demonstrated that C-LDA is insensitive to hyperparameters and works well in both multi-domain and single-domain adaptation scenarios.

Unsupervised Speaker Adaptation Of Deep Neural Network Based On The Combination Of Speaker Codes And Singular Value Decomposition For Speech Recognition

Fast Adaptation of Deep Neural Network Based on Discriminant Codes for Speech Recognition

Speaker Adaptation of Hybrid NN/HMM Model for Speech Recognition Based on Singular Value Decomposition

A Unified Speaker Adaptation Method for Speech Synthesis using Transcribed and Untranscribed Speech with Backpropagation

Bayesian Learning for Deep Neural Network Adaptation

Codebook-Based Speaker Adaptation

Speaker-Smoothed kNN Speaker Adaptation for End-to-End ASR

Online Speaker Adaptation for LVCSR Based on Attention Mechanism

Online Speaker Adaptation Using Memory-Aware Networks for Speech Recognition

Linear Networks Based Speaker Adaptation for Speech Synthesis

Investigating Online Low-Footprint Speaker Adaptation Using Generalized Linear Regression and Click-Through Data.

Batch Normalization based Unsupervised Speaker Adaptation for Acoustic Models

An New Approach for Incremental Speaker Adaptation

Scaling and bias codes for modeling speaker-adaptive DNN-based speech synthesis systems

Speaker Adaptation Using Spectro-Temporal Deep Features for Dysarthric and Elderly Speech Recognition

A Speaker-Dependent Deep Learning Approach to Joint Speech Separation and Acoustic Modeling for Multi-Talker Automatic Speech Recognition

Speaker Normalization Training and Adaptation for Speech Recognition

Supervised And Unsupervised Speaker Adaptation Using Confidence Measure

Extended Low-Rank Plus Diagonal Adaptation for Deep and Recurrent Neural Networks.

Efficient Black-Box Speaker Verification Model Adaptation with Reprogramming and Backend Learning

A Simple Unsupervised Knowledge-Free Domain Adaptation for Speaker Recognition