Abstract:The extraction of a target speaker from mixtures of different speakers has attracted extensive amounts of attention and research. Previous studies have proposed several methods, such as SpeakerBeam, to tackle this speech extraction problem using clean speech from the target speaker to provide information. However, clean speech cannot be obtained immediately in most cases. In this study, we addressed this problem by extracting features from the electroglottographs (EGGs) of target speakers. An EGG is a laryngeal function detection technology that can detect the impedance and condition of vocal cords. Since EGGs have excellent anti-noise performance due to the collection method, they can be obtained in rather noisy environments. In order to obtain clean speech from target speakers out of the mixtures of different speakers, we utilized deep learning methods and used EGG signals as additional information to extract target speaker. In this way, we could extract target speaker from mixtures of different speakers without needing clean speech from the target speakers. According to the characteristics of the EGG signals, we developed an EGG_auxiliary network to train a speaker extraction model under the assumption that EGG signals carry information about speech signals. Additionally, we took the correlations between EGGs and speech signals in silent and unvoiced segments into consideration to develop a new network involving EGG preprocessing. We achieved improvements in the scale invariant signal-to-distortion ratio improvement (SISDRi) of 0.89 dB on the Chinese Dual-Mode Emotional Speech Database (CDESD) and 1.41 dB on the EMO-DB dataset. In addition, our methods solved the problem of poor performance with target speakers of the same gender and the different between the same gender situation and the problem of greatly reduced precision under the low SNR circumstances.

New Insights on Target Speaker Extraction

Focus on the Sound around You: Monaural Target Speaker Extraction via Distance and Speaker Information

Target Speech Extraction Based on Blind Source Separation and X-vector-based Speaker Selection Trained with Data Augmentation

Audio-Visual Target Speaker Extraction with Reverse Selective Auditory Attention

An Electroglottograph Auxiliary Neural Network for Target Speaker Extraction

Speaker-conditioning Single-channel Target Speaker Extraction using Conformer-based Architectures

Dual-Channel Target Speaker Extraction Based on Conditional Variational Autoencoder and Directional Information

USEF-TSE: Universal Speaker Embedding Free Target Speaker Extraction

Probing Self-supervised Learning Models with Target Speech Extraction

3S-TSE: Efficient Three-Stage Target Speaker Extraction for Real-Time and Low-Resource Applications

Improving Target Speaker Extraction with Sparse LDA-transformed Speaker Embeddings

Speaker Extraction with Detection of Presence and Absence of Target Speakers

Target Speech Extraction with Pre-trained Self-supervised Learning Models

Improving Source Separation via Multi-Speaker Representations

Time-Domain Speech Extraction with Spatial Information and Multi Speaker Conditioning Mechanism

Target Speaker Extraction by Directly Exploiting Contextual Information in the Time-Frequency Domain

X-CrossNet: A complex spectral mapping approach to target speaker extraction with cross attention speaker embedding fusion

Combination of Multiple Embeddings for Speaker Retrieval

X-SepFormer: End-to-end Speaker Extraction Network with Explicit Optimization on Speaker Confusion

All Information is Necessary: Integrating Speech Positive and Negative Information by Contrastive Learning for Speech Enhancement

Audio-Visual Active Speaker Extraction for Sparsely Overlapped Multi-talker Speech