Abstract:Speaker representation learning is critical for modern voice recognition systems. While supervised learning techniques require extensive labeled data, unsupervised methodologies can leverage vast unlabeled corpora, offering a scalable solution. This paper introduces self-supervised reflective learning (SSRL), a novel paradigm that streamlines existing iterative unsupervised frameworks. SSRL integrates self-supervised knowledge distillation with online clustering to refine pseudo labels and train the model without iterative bottlenecks. Specifically, a teacher model continually refines pseudo labels through online clustering, providing dynamic supervision signals to train the student model. The student model undergoes noisy student training with input and model noise to boost its modeling capacity. The teacher model is updated via an exponential moving average of the student, acting as an ensemble of past iterations. Further, a pseudo label queue retains historical labels for consistency, and noisy label modeling directs learning towards clean samples. Experiments on VoxCeleb show SSRL's superiority over current iterative approaches, surpassing the performance of a 5-round method in just a single training round. Ablation studies validate the contributions of key components like noisy label modeling and pseudo label queues. Moreover, consistent improvements in pseudo labeling and the convergence of cluster counts demonstrate SSRL's effectiveness in deciphering unlabeled data. This work marks an important advancement in efficient and accurate speaker representation learning through the novel reflective learning paradigm.

Self-supervised Representation Learning for Speech Processing

Self-Supervised Speech Representation Learning: A Review

Editorial Editorial of Special Issue on Self-Supervised Learning for Speech and Audio Processing

Sustainable self-supervised learning for speech representations

Self Supervised Representation Learning with Deep Clustering for Acoustic Unit Discovery from Raw Speech

Efficiency-oriented approaches for self-supervised speech representation learning

Self-supervised Reflective Learning through Self-distillation and Online Clustering for Speaker Representation Learning

Representation Learning With Hidden Unit Clustering For Low Resource Speech Applications

Deep Representation Learning in Speech Processing: Challenges, Recent Advances, and Future Trends

Progressive Multi-scale Self-supervised Learning for Speech Recognition

Speech Representation Learning Through Self-supervised Pretraining And Multi-task Finetuning

Learning Speech Representations from Raw Audio by Joint Audiovisual Self-Supervision

SUPERB @ SLT 2022: Challenge on Generalization and Efficiency of Self-Supervised Speech Representation Learning

Self-Supervised Representation Learning: Introduction, advances, and challenges

Understanding Self-Supervised Learning of Speech Representation via Invariance and Redundancy Reduction

Investigating Self-Supervised Learning for Speech Enhancement and Separation

An Exploration of Self-Supervised Pretrained Representations for End-to-End Speech Recognition.

An Exploration of Self-Supervised Pretrained Representations for End-to-End Speech Recognition

SUPERB: Speech Understanding and PERformance Benchmark

Progressive Residual Extraction based Pre-training for Speech Representation Learning

Bootstrapped Unsupervised Sentence Representation Learning.