Abstract:Deep speaker embedding learning based on neural networks has become the predominant approach in speaker verification (SV) currently. In prior studies, researchers have investigated various network architectures. However, rare works pay attention to the question of how to achieve a better trade-off on model performance and computational complexity. In this paper, we focus on efficient architecture design for speaker verification. Firstly, we systematically study the effect of the network depth and width on performance and empirically discover that depth is more important than the width of networks for speaker verification task. Based on this observation, we propose a novel depth-first (DF) architecture design rule. By applying it to ResNet and ECAPA-TDNN, two new families of much deeper models, namely DF-ResNets and DF-ECAPAs, are constructed. In addition, to further boost the performance of small models in the low computation regime, two novel attentive feature fusion (AFF) schemes, including sequential AFF (S-AFF) and parallel AFF (P-AFF), are proposed to dynamically fuse features in a learnable way. Experimental results on the VoxCeleb dataset show that the newly proposed DF-ResNets and DF-ECAPAs can achieve a much better trade-off on performance and complexity than the original ResNet and ECAPA-TDNN. Moreover, small models can further obtain up to 40% relative improvement in EER by adopting AFF scheme with negligible computational cost. Finally, a comprehensive comparison with various other published SV systems illustrates that our proposed models achieve the best trade-off on performance and complexity in both low and high computation scenarios.

Self-Knowledge Distillation via Feature Enhancement for Speaker Verification

Knowledge Distillation For Small Foot-Print Deep Speaker Embedding

Emphasized Non-Target Speaker Knowledge in Knowledge Distillation for Automatic Speaker Verification

Integrated Multi-Level Knowledge Distillation for Enhanced Speaker Verification

Distilling Multi-Level X-vector Knowledge for Small-footprint Speaker Verification

One-Step Knowledge Distillation and Fine-Tuning in Using Large Pre-Trained Self-Supervised Learning Models for Speaker Verification

Leveraging ASR Pretrained Conformers for Speaker Verification through Transfer Learning and Knowledge Distillation

Distil-DCCRN: A Small-footprint DCCRN Leveraging Feature-based Knowledge Distillation in Speech Enhancement

End-to-End Feature Learning for Text-Independent Speaker Verification

Self-Knowledge Distillation in Natural Language Processing

Speaker Verification With Deep Features

Depth-First Neural Architecture with Attentive Feature Fusion for Efficient Speaker Verification.

Knowledge Distillation from Multiple Foundation Models for End-to-End Speech Recognition

Self-Distillation: Towards Efficient and Compact Neural Networks

Self-boosting for Feature Distillation

Lightweight Self-Knowledge Distillation with Multi-source Information Fusion

Self-Distillation Prototypes Network: Learning Robust Speaker Representations without Supervision

DF-ResNet: Boosting Speaker Verification Performance with Depth-First Design

Feature Fusion-Based Collaborative Learning for Knowledge Distillation.