SALT: Distinguishable Speaker Anonymization Through Latent Space Transformation

Yuanjun Lv,Jixun Yao,Peikun Chen,Hongbin Zhou,Heng Lu,Lei Xie

2023-10-08

Abstract:Speaker anonymization aims to conceal a speaker's identity without degrading speech quality and intelligibility. Most speaker anonymization systems disentangle the speaker representation from the original speech and achieve anonymization by averaging or modifying the speaker representation. However, the anonymized speech is subject to reduction in pseudo speaker distinctiveness, speech quality and intelligibility for out-of-distribution speaker. To solve this issue, we propose SALT, a Speaker Anonymization system based on Latent space Transformation. Specifically, we extract latent features by a self-supervised feature extractor and randomly sample multiple speakers and their weights, and then interpolate the latent vectors to achieve speaker anonymization. Meanwhile, we explore the extrapolation method to further extend the diversity of pseudo speakers. Experiments on Voice Privacy Challenge dataset show our system achieves a state-of-the-art distinctiveness metric while preserving speech quality and intelligibility. Our code and demo is availible at <a class="link-external link-https" href="https://github.com/BakerBunker/SALT" rel="external noopener nofollow">this https URL</a> .

Sound,Audio and Speech Processing

What problem does this paper attempt to address?

The paper aims to address several key issues in the process of voice anonymization:

1. **Enhancing the uniqueness of anonymized voices**: Existing anonymization methods suffer from limited sampling space when dealing with out-of-distribution speakers, resulting in insufficient diversity in the timbre of pseudo-speakers and reduced uniqueness of anonymized voices.

2. **Maintaining voice quality and intelligibility**: Current methods significantly degrade voice quality when handling out-of-distribution speakers, especially when the randomly generated pseudo-speaker vectors are far from the training set distribution.

3. **Information loss and leakage**: Methods based on automatic speech recognition (ASR) and speaker verification (SV) models inevitably lead to information loss or leakage when separating content and speaker information, thereby affecting speaker uniqueness and audio quality.

To address these issues, the authors propose a voice anonymization system based on latent space transformation—SALT. This system extracts latent features through a self-supervised feature extractor (such as WavLM) and achieves speaker anonymization using a random weighted average method. It also introduces interpolation and extrapolation methods to increase the diversity of pseudo-speakers. Experimental results show that SALT significantly enhances the uniqueness of anonymized voices while maintaining voice quality and intelligibility.