Abstract:Deep speaker embedding learning based on neural networks has become the predominant approach in speaker verification (SV) currently. In prior studies, researchers have investigated various network architectures. However, rare works pay attention to the question of how to achieve a better trade-off on model performance and computational complexity. In this paper, we focus on efficient architecture design for speaker verification. Firstly, we systematically study the effect of the network depth and width on performance and empirically discover that depth is more important than the width of networks for speaker verification task. Based on this observation, we propose a novel depth-first (DF) architecture design rule. By applying it to ResNet and ECAPA-TDNN, two new families of much deeper models, namely DF-ResNets and DF-ECAPAs, are constructed. In addition, to further boost the performance of small models in the low computation regime, two novel attentive feature fusion (AFF) schemes, including sequential AFF (S-AFF) and parallel AFF (P-AFF), are proposed to dynamically fuse features in a learnable way. Experimental results on the VoxCeleb dataset show that the newly proposed DF-ResNets and DF-ECAPAs can achieve a much better trade-off on performance and complexity than the original ResNet and ECAPA-TDNN. Moreover, small models can further obtain up to 40% relative improvement in EER by adopting AFF scheme with negligible computational cost. Finally, a comprehensive comparison with various other published SV systems illustrates that our proposed models achieve the best trade-off on performance and complexity in both low and high computation scenarios.

Joint speaker encoder and neural back-end model for fully end-to-end automatic speaker verification with multiple enrollment utterances

An Effective Speaker Recognition Method Based on Joint Identification and Verification Supervisions.

Post-Training Embedding Alignment for Decoupling Enrollment and Runtime Speaker Recognition Models

Bidirectional Attention For Text-Dependent Speaker Verification

End-to-End Feature Learning for Text-Independent Speaker Verification

End-to-End Speaker Verification Via Curriculum Bipartite Ranking Weighted Binary Cross-Entropy

Jointing Multi-task Learning and Gradient Reversal Layer for Far-Field Speaker Verification

Depth-First Neural Architecture with Attentive Feature Fusion for Efficient Speaker Verification.

Learning Discriminative Speaker Embedding by Improving Aggregation Strategy and Loss Function for Speaker Verification

Joint Feature Enhancement and Speaker Recognition with Multi-Objective Task-Oriented Network.

Towards Robust Speaker Verification with Target Speaker Enhancement

Contrastive Learning for improving End-to-end Speaker Verification

Joint I-Vector with End-to-End System for Short Duration Text-Independent Speaker Verification.

Joint Speaker Features Learning for Audio-visual Multichannel Speech Separation and Recognition

A speaker verification backend with robust performance across conditions

A joint training framework of multi-look separator and speaker embedding extractor for overlapped speech

EEND-SS: Joint End-to-End Neural Speaker Diarization and Speech Separation for Flexible Number of Speakers

Automatic enrollment for speaker authentication.

ECAPA2: A Hybrid Neural Network Architecture and Training Strategy for Robust Speaker Embeddings

Jointly Adversarial Enhancement Training for Robust End-to-End Speech Recognition

End-to-End Attention based Text-Dependent Speaker Verification