Abstract:Speaker embeddings represent a means to extract representative vectorial representations from a speech signal such that the representation pertains to the speaker identity alone. The embeddings are commonly used to classify and discriminate between different speakers. However, there is no objective measure to evaluate the ability of a speaker embedding to disentangle the speaker identity from the other speech characteristics. This means that the embeddings are far from ideal, highly dependent on the training corpus and still include a degree of residual information pertaining to factors such as linguistic content, recording conditions or speaking style of the utterance. This paper introduces an analysis over six sets of speaker embeddings extracted with some of the most recent and high-performing DNN architectures, and in particular, the degree to which they are able to truly disentangle the speaker identity from the speech signal. To correctly evaluate the architectures, a large multi-speaker parallel speech dataset is used. The dataset includes 46 speakers uttering the same set of prompts, recorded in either a professional studio or their home environments. The analysis looks into the intra- and inter-speaker similarity measures computed over the different embedding sets, as well as if simple classification and regression methods are able to extract several residual information factors from the speaker embeddings. The results show that the discriminative power of the analyzed embeddings is very high, yet across all the analyzed architectures, residual information is still present in the representations in the form of a high correlation to the recording conditions, linguistic contents and utterance duration.

On Feature Importance and Interpretability of Speaker Representations

Investigating Speaker Embedding Disentanglement on Natural Read Speech

Disentangling Textual and Acoustic Features of Neural Speech Representations

Residual Information in Deep Speaker Embedding Architectures

An empirical analysis of information encoded in disentangled neural speaker representations

Explaining Deep Learning Embeddings for Speech Emotion Recognition by Predicting Interpretable Acoustic Features

Powerful Speaker Embedding Training Framework by Adversarially Disentangled Identity Representation

Self-Supervised Disentangled Representation Learning for Robust Target Speech Extraction

Probing Deep Speaker Embeddings for Speaker-related Tasks

Exploiting Speaker Embeddings for Improved Microphone Clustering and Speech Separation in ad-hoc Microphone Arrays

An Information-Theoretic Analysis of Self-supervised Discrete Representations of Speech

What Does The Speaker Embedding Encode?

Speaker and Style Disentanglement of Speech Based on Contrastive Predictive Coding Supported Factorized Variational Autoencoder

An analysis on the effects of speaker embedding choice in non auto-regressive TTS

Disentangled Representation Learning for Environment-agnostic Speaker Recognition

On the Usage of Phonetic Information for Text-Independent Speaker Embedding Extraction.

Speaker Embedding Extraction with Phonetic Information

What Do Speech Foundation Models Not Learn About Speech?

Overview of Speaker Modeling and Its Applications: From the Lens of Deep Speaker Representation Learning

Quantitative Evidence on Overlooked Aspects of Enrollment Speaker Embeddings for Target Speaker Separation

Which to select?: Analysis of speaker representation with graph attention networks