Abstract:Automatic speaker verification task has made great achievements using deep learning approaches with the large-scale manually annotated dataset. However, it's very difficult and expensive to collect a large amount of well-labeled data for system building. In this paper, we propose a novel and advanced self-supervised learning framework which can construct a high performance speaker verification system without using any labeled data. To avoid the impact of false negative pairs, we adopt the self-distillation with no labels (DINO) framework as the initial model, which can be trained without exploiting negative pairs. Then, we introduce a cluster-aware training strategy for DINO to improve the diversity of data. In the iteration learning stage, due to a mass of unreliable labels from clustering, the quality of pseudo labels is important for the system training. This motivates us to propose dynamic loss-gate and label correction (DLG-LC) methods to alleviate the performance degradation caused by unreliable labels. More specifically, we model the loss distribution with GMM and obtain the loss-gate threshold dynamically to distinguish the reliable and unreliable labels. Besides, we adopt the model predictions to correct the unreliable label, for better utilizing the unreliable data rather than dropping them directly. Moreover, we extend the DLG-LC to multi-modality to further improve the performance. The experiments are performed on the commonly used Voxceleb dataset. Compared to the best-known self-supervised speaker verification system, our proposed method obtain 22.17%, 27.94% and 25.56% relative EER improvement on Vox-O, Vox-E and Vox-H test sets, even with fewer iterations, smaller models, and simpler clustering methods. More importantly, the newly proposed system even achieves comparable results with the fully supervised system, but without using any human labeled data.

Large-Scale Self-Supervised Speech Representation Learning for Automatic Speaker Verification

Large-Scale Self-Supervised Speech Representation Learning for Automatic Speaker Verification

VarASV: Enabling Pitch-variable Automatic Speaker Verification Via Multi-task Learning

Towards Supervised Performance on Speaker Verification with Self-Supervised Learning by Leveraging Large-Scale ASR Models

Self-Supervised Learning Based Domain Adaptation for Robust Speaker Verification

Self-Supervised Learning with Cluster-Aware-DINO for High-Performance Robust Speaker Verification

A Comprehensive Study on Self-Supervised Distillation for Speaker Representation Learning

Why Does Self-Supervised Learning for Speech Recognition Benefit Speaker Recognition?

Self-Distillation Prototypes Network: Learning Robust Speaker Representations without Supervision

Asymmetric Clean Segments-Guided Self-Supervised Learning for Robust Speaker Verification

Label-Efficient Self-Supervised Speaker Verification With Information Maximization and Contrastive Learning

Dual-model self-regularization and fusion for domain adaptation of robust speaker verification

An Effective Speaker Recognition Method Based on Joint Identification and Verification Supervisions.

The SpeakIn Speaker Verification System for Far-Field Speaker Verification Challenge 2022

BigSSL: Exploring the Frontier of Large-Scale Semi-Supervised Learning for Automatic Speech Recognition

Auto-AVSR: Audio-Visual Speech Recognition with Automatic Labels

An Exploration of Self-Supervised Pretrained Representations for End-to-End Speech Recognition

An Exploration of Self-Supervised Pretrained Representations for End-to-End Speech Recognition.

Pushing the limits of self-supervised speaker verification using regularized distillation framework

Speaker Verification with Disentangled Self-attention.