Detection and reconstruction of clipped speech for speaker recognition

Fanhu Bie,Dong Wang,Jun Wang,Thomas Fang Zheng

DOI: https://doi.org/10.1016/j.specom.2015.06.008

IF: 2.723

2015-01-01

Speech Communication

Abstract:•The impact of speech clipping on speaker recognition was investigated.•A simple yet efficient clipping detection method was proposed.•A novel DNN-based clipping reconstruction approach was presented.

What problem does this paper attempt to address?

Research on Speaker-Depended Isolated-Word Speech Recognition System

Jianguo Lü

2011-01-01

Abstract:Voice-identification is a kind of technology that is using computer to transfer the voice signal to an associated text or command by identification and understand.Zero-crossing rate and short-term energy are selected as the basis in the endpoint detecting in preprocessing of speech signal through the studying of the characters of single-word.LPC(Linear Prediction Coefficient)is extracted from the signal,then cepstral coefficient is obtained as the speech characteristic parameters.DTW(Dynamic Time Warping)algorithm,which is improved to reduce the amount of data in the matching process by using global constraint,is used for the matching of the model.Based on the algorithm,a speaker-depended isolated-word speech recognition system is designed,and the simulation and analysis is carried on by using Matlab in a variety of background conditions.The experiment shows that the algorithm for speaker-depended isolated-word speech recognition can achieve good results.
A Novel and Efficient Voice Activity Detector Using Shape Features of Speech Wave.

Qiming Zhao,Yingchun Yang,Hong Li

DOI: https://doi.org/10.1007/978-3-319-12484-1_42

2014-01-01

Abstract:A voice activity detector (VAD) is the prerequisite for speaker recognition in real life. Currently, we deal with the VAD problem at the frame level through short time window function. However, when tackling with the VAD problem manually, we can easily pick out the speech segments containing several words. Inspired by this, we firstly use IIR filter to get the envelope of the waveform and divide the envelope into separate sound segments. And then we extract shape features from the obtained segments and use K-means to cluster the data featured by the amplitude of the wave crest to discard the silent part. Finally, we utilize other shape features to discard the noise part. The performance of our proposed VAD method has apparently surpassed the energy-based VAD and VQVAD with a relative 20% decrease in error rate, While the computation time of the proposed VAD method is only 30% less than that of VQVAD. We also get an encouraging result utilizing our VAD method for speaker recognition with about 3% average decrease in EER.
Self-attention Based Speaker Recognition Using Cluster-Range Loss

Tengyue Bian,Fangzhou Chen,Li Xu

DOI: https://doi.org/10.1016/j.neucom.2019.08.046

IF: 6

2019-01-01

Neurocomputing

Abstract:Speaker recognition with short utterances is a challenging research topic in the natural language processing (NLP) field. Previous convolutional neural network (CNN) based models for speaker recognition usually utilize very deep or wide layers, resulting in many parameters and high computational cost. Besides, great training difficulty and inefficiency exist in the triplet loss, which is widely used in speaker recognition. In this work, we propose to combine the residual network (ResNet) with the self-attention mechanism to achieve better performance in text-independent speaker recognition with fewer parameters and lower computational cost. In addition, the Cluster-Range Loss based on a well-designed online exemplar mining is proposed to directly shrink the intra-class variation and to enlarge the inter-class distance. Experiments on Voxceleb dataset are conducted to verify the effectiveness of the proposed scheme. The proposed approach achieves a Top-1 accuracy of 89.1% for speaker identification by jointly training the network with the Cluster-Range Loss and softmax cross entropy loss. For speaker verification, we achieve a competitive EER of 5.5% without any heavy-tailed backend, compared with the state-of-the-art i-vector system, as well as the x-vector system.
DNN-based Voice Activity Detection for Speaker Recognition

Fanhu Bie,Zhiyong Zhang,Dong Wang,Thomas Fang Zheng

2015-01-01

Abstract:Correspondence: biefh@cslt.riit.tsinghua.edu.cn Center for Speech and Language Technology, Research Institute of Information Technology, Tsinghua University, ROOM 1-303, BLDG FIT, 100084 Beijing, China Department of Computer Science and Technology, Tsinghua University, ROOM 1-303, BLDG FIT, 100084 Beijing, China Full list of author information is available at the end of the article Abstract Voice activity detection (VAD) plays an important role in speaker recognition. This paper proposes to use a novel DNN-based VAD which harnesses the power of deep neural networks (DNN) in learning speech patterns from a large labelled database designed for speech recognition, and thus deliberately optimizes the discrimination between speech and non-speech signals. More interestingly, the output of the DNN offers a noise prior, which may lend itself to a Bayesian treatment for the uncertainty of noise in speaker recognition. The experiments were conducted on the mismatched-microphone condition (C3) of the SRE08 core test. It was found that the DNN-based VAD offered a relative reduction of 22.0% in equal error rate (EER) when compared to a fine-tuned energy-based VAD. When the Bayesian approach was employed, additional gains were obtained, particularly in noise conditions.
DDD: A Perceptually Superior Low-Response-Time DNN-based Declipper

Jayeon Yi,Junghyun Koo,Kyogu Lee

2024-01-08

Abstract:Clipping is a common nonlinear distortion that occurs whenever the input or output of an audio system exceeds the supported range. This phenomenon undermines not only the perception of speech quality but also downstream processes utilizing the disrupted signal. Therefore, a real-time-capable, robust, and low-response-time method for speech declipping (SD) is desired. In this work, we introduce DDD (Demucs-Discriminator-Declipper), a real-time-capable speech-declipping deep neural network (DNN) that requires less response time by design. We first observe that a previously untested real-time-capable DNN model, Demucs, exhibits a reasonable declipping performance. Then we utilize adversarial learning objectives to increase the perceptual quality of output speech without additional inference overhead. Subjective evaluations on harshly clipped speech shows that DDD outperforms the baselines by a wide margin in terms of speech quality. We perform detailed waveform and spectral analyses to gain an insight into the output behavior of DDD in comparison to the baselines. Finally, our streaming simulations also show that DDD is capable of sub-decisecond mean response times, outperforming the state-of-the-art DNN approach by a factor of six.

Audio and Speech Processing,Sound,Signal Processing
Speaker Recognition Based on Pre-Trained Model and Deep Clustering

Liang He,Zhida Song,Shuanghong Liu,Mengqi Niu,Ying Hu,Hao Huang

DOI: https://doi.org/10.1109/icme57554.2024.10687367

2024-01-01

Abstract:In this paper, we propose a novel loss by integrating a deep clustering (DC) loss at the frame-level and a speaker recognition loss at the segment-level into a single network without additional data requirements and exhaustive computation. The DC loss implicitly generates soft pseudo-phoneme labels for each frame-level feature, which facilitates extracting more discriminant speaker representation by suppressing phonetic content information. We study the DC loss not only on the acoustic feature, but also on the features extracted by the pre-trained models, such as wav2vec 2.0, HuBERT and WavLM. Experimental results on the VoxCeleb dataset shows that the overall system performance based on the pre-trained model features are better than the one on the acoustic feature. The proposed loss is significantly effective for systems on the acoustic feature and has a marginal improvement for systems on the pre-trained model feature.
Research on truncated speech in speaker verification

Fanhu Bie,Dong Wang,Thomas Fang Zheng

DOI: https://doi.org/10.1109/ISCSLP.2014.6936671

2014-01-01

Abstract:Summary form only given. The speech truncating phenomenon is a general problem is practical speaker recognition system. After the speech was truncated by amplitude, the spectral was changed during the process, resulting in the decreasing in the system`s performance. The paper describes the observation and the conclusion on the impact of the truncated segments, studies the reason of the impact on the recognition performance, gives out the ways of the truncated segments detection and reducing the decreasing of the performance. The simulation on NIST SRE08 shows that, just when the amplitude truncating ratio remains high (more than the 80% of the maximum amplitude), the performance drops sharply; the performance of traditional GMM-UBM system and I-vector system behavior familiar when the amplitude truncating is low, while I-vector gives a better robustness when is high. The paper gives out a proposal on truncating segments detection based on subspace discriminant information, which is then used to discard the truncating segments. The experiments show that this proposal could well detect the truncated segments. However, the results show that there are still speaker discriminant information in the truncated segments, when the amplitude truncated ratio remains low, it's better to remain the data to sustain the performance, otherwise, the speaker should take another recording to keep the system performance.
Speech-Declipping Transformer with Complex Spectrogram and Learnerble Temporal Features

Younghoo Kwon,Jung-Woo Choi

2024-09-19

Abstract:We present a transformer-based speech-declipping model that effectively recovers clipped signals across a wide range of input signal-to-distortion ratios (SDRs). While recent time-domain deep neural network (DNN)-based declippers have outperformed traditional handcrafted and spectrogram-based DNN approaches, they still struggle with low-SDR inputs. To address this, we incorporate a transformer-based architecture that operates in the time-frequency (TF) domain. The TF-transformer architecture has demonstrated remarkable performance in the speech enhancement task for low-SDR signals but cannot be optimal for the time-domain artifact like clipping. To overcome the limitations of spectrogram-based DNNs, we design an extra convolutional block that directly extracts temporal features from time-domain waveforms. The joint analysis of complex spectrogram and learned temporal features allows the model to improve performance on both high- and low-SDR inputs. Our approach also preserves the unclipped portions of the speech signal during processing, preventing degradation typically seen when only spectral information is used. In evaluations on the VoiceBank-DEMAND and DNS challenge datasets, the proposed model consistently outperformed state-of-the-art (SOTA) declipping models across various metrics, demonstrating its robustness and generalizability.

Audio and Speech Processing,Sound,Signal Processing
Weighted Cluster-Range Loss and Criticality-Enhancement Loss for Speaker Recognition

Jianye Mo,Li Xu

DOI: https://doi.org/10.3390/app10249004

2020-01-01

Abstract:While traditional i-vector based methods are popular in the field of speaker recognition, deep learning has recently found more and more applications to the end-to-end models due to its attractive performance. One effective practice is the integration of attention mechanism into the Convolution Neural Networks (CNNs). In this work, a light-weight dual-path attention block is proposed by combining the self-attention and Convolutional Block Attention Module (CBAM), which helps to capture more multi-source features with neglectable extra time expense. Additionally, a Weighted Cluster-Range Loss (WCRL) is proposed to enhance the identification performance of Cluster-Range Loss (CRL) on indecisive samples. Besides, to address the low efficiency in the initial training stage of CRL, a novel Criticality-Enhancement Loss (CEL) is also presented. Both of the proposed loss functions could significantly promote the training efficiency and globally improve the recognition performance. Experimental results are presented to show the effectiveness of the proposed scheme, which achieves a competitive top-1 accuracy of 92.0%, top-5 accuracy of 97.6%, and Equal Error Rate (EER) of 3.5% on the VoxCeleb1 dataset.
Analysis of Deep Clustering as Preprocessing for Automatic Speech Recognition of Sparsely Overlapping Speech

Tobias Menne,Ilya Sklyar,Ralf Schlüter,Hermann Ney

DOI: https://doi.org/10.21437/Interspeech.2019-1728

2019-09-25

Abstract:Significant performance degradation of automatic speech recognition (ASR) systems is observed when the audio signal contains cross-talk. One of the recently proposed approaches to solve the problem of multi-speaker ASR is the deep clustering (DPCL) approach. Combining DPCL with a state-of-the-art hybrid acoustic model, we obtain a word error rate (WER) of 16.5 % on the commonly used wsj0-2mix dataset, which is the best performance reported thus far to the best of our knowledge. The wsj0-2mix dataset contains simulated cross-talk where the speech of multiple speakers overlaps for almost the entire utterance. In a more realistic ASR scenario the audio signal contains significant portions of single-speaker speech and only part of the signal contains speech of multiple competing speakers. This paper investigates obstacles of applying DPCL as a preprocessing method for ASR in such a scenario of sparsely overlapping speech. To this end we present a data simulation approach, closely related to the wsj0-2mix dataset, generating sparsely overlapping speech datasets of arbitrary overlap ratio. The analysis of applying DPCL to sparsely overlapping speech is an important interim step between the fully overlapping datasets like wsj0-2mix and more realistic ASR datasets, such as CHiME-5 or AMI.

Sound,Computation and Language,Audio and Speech Processing
Speaker recognition based on improved ECAPA-TDNN network

Xiaoming Ling,Zeifei Dang,Hongyan Chen,Xiang Ji,Zhen Zhang,Xiaoyu Zhang

DOI: https://doi.org/10.1117/12.2689511

2023-08-10

Abstract:Speaker recognition, also known as voiceprint recognition, is a biometric technology with wide practicability at present. This paper summarizes and compares and analyzes the main research methods of speaker recognition at home and abroad at this stage, and proposes an improved ECAPA_TDNN algorithm. It is proved by experiments that the improved ECAPA_TDNN algorithm in this paper is superior to the classical algorithm in terms of accuracy and loss.

Computer Science,Engineering
SPEECHCLIP: INTEGRATING SPEECH WITH PRE-TRAINED VISION AND LANGUAGE MODEL

Yi-Jen Shih,Hsuan-Fu Wang,Heng-Jui Chang,Layne Berry,Hung-yi Lee,David Harwath

DOI: https://doi.org/10.1109/slt54892.2023.10022954

2022-01-01

Abstract:Data-driven speech processing models usually perform well with a large amount of text supervision, but collecting transcribed speech data is costly. Therefore, we propose SpeechCLIP, a novel framework bridging speech and text through images to enhance speech models without transcriptions. We leverage state-of-the-art pre-trained HuBERT and CLIP, aligning them via paired images and spoken captions with minimal fine-tuning. SpeechCLIP outperforms prior stateof-the-art on image-speech retrieval and performs zero-shot speech-text retrieval without direct supervision from transcriptions. Moreover, SpeechCLIP can directly retrieve semantically related keywords from speech.
Speaker Clustering Algorithm in Speech Recognition

XIAO Shu-cai,Ou Zhi-jian,WANG Zuo-ying

DOI: https://doi.org/10.3969/j.issn.1003-0077.2005.04.013

2005-01-01

Abstract:In this paper, We introduced a speaker clustering algorithm in speech recognition, which includes its effect to the recognition system. Also, its usage, the features used, distance measurement and the procedure of the algorithm were described. To evaluate the effectiveness of the algorithm, we do two kinds of experiments. One is by calculating the clustering correction rate directly and the other is by comparing the word error rate (WER) of the recognition system under two different conditions: whether using the speaker clustering algorithm or not. From the experiments, we can see that the sentence clustering correction rate is reached 85.69% when using the GLR distance measurement.In the recognition experiment, the performance of the system improves a lot, that the word error rate is very near that of the system by using the known speaker information to do the speaker adaptation.
DNN based Speaker Recognition on Short Utterances

Ahilan Kanagasundaram,David Dean,Sridha Sridharan,Clinton Fookes

DOI: https://doi.org/10.48550/arXiv.1610.03190

2016-10-11

Sound

Abstract:This paper investigates the effects of limited speech data in the context of speaker verification using deep neural network (DNN) approach. Being able to reduce the length of required speech data is important to the development of speaker verification system in real world applications. The experimental studies have found that DNN-senone-based Gaussian probabilistic linear discriminant analysis (GPLDA) system respectively achieves above 50% and 18% improvements in EER values over GMM-UBM GPLDA system on NIST 2010 coreext-coreext and truncated 15sec-15sec evaluation conditions. Further when GPLDA model is trained on short-length utterances (30sec) rather than full-length utterances (2min), DNN-senone GPLDA system achieves above 7% improvement in EER values on truncated 15sec-15sec condition. This is because short length development i-vectors have speaker, session and phonetic variation and GPLDA is able to robustly model those variations. For several real world applications, longer utterances (2min) can be used for enrollment and shorter utterances (15sec) are required for verification, and in those conditions, DNN-senone GPLDA system achieves above 26% improvement in EER values over GMM-UBM GPLDA systems.
Efficient Identification Of Speakers In News Video Based On Shot Segmentation

Qing Chang,Xiangyang Xue,Hong Lu,You-san Nie

DOI: https://doi.org/10.1109/ICARCV.2004.1469078

2004-01-01

Abstract:An effective method for speaker identification in news video is presented in this paper, which is based on shot segmentation and exploits both audio and visual cues. Firstly, audio is segmented by shot segmentation based on the observation that there is only one speaker in a shot of news video in most cases. Furthermore, speech/non-speech discrimination is implemented on each shot. Finally, text-independent speaker identification is proposed using audio features on the discriminated speech shots. Experimental results show that our algorithm can obtain satisfactory performance in identifying speakers, so it can be used in real application.
Pyramid Feature Attention Network for Speech Resampling Detection

Xinyu Zhou,Yujin Zhang,Yongqi Wang,Jin Tian,Shaolun Xu

DOI: https://doi.org/10.3390/app14114803

2024-01-01

Abstract:Speech forgery and tampering, increasingly facilitated by advanced audio editing software, pose significant threats to the integrity and privacy of digital speech avatars. Speech resampling is a post-processing operation of various speech-tampering means, and the forensic detection of speech resampling is of great significance. For speech resampling detection, most of the previous works used traditional methods of feature extraction and classification to distinguish original speech from forged speech. In view of the powerful ability of deep learning to extract features, this paper converts the speech signal into a spectrogram with time-frequency characteristics, and uses the feature pyramid network (FPN) with the Squeeze and Excitation (SE) attention mechanism to learn speech resampling features. The proposed method combines the low-level location information and the high-level semantic information, which dramatically improves the detection performance of speech resampling. Experiments were carried out on a resampling corpus made on the basis of the TIMIT dataset. The results indicate that the proposed method significantly improved the detection accuracy of various resampled speech. For the tampered speech with a resampling factor of 0.9, the detection accuracy is increased by nearly 20%. In addition, the robustness test demonstrates that the proposed model has strong resistance to MP3 compression, and the overall performance is better than the existing methods.
Modified layer deep convolution neural network for text-independent speaker recognition

V Karthikeyan,Suja Priyadharsini S

DOI: https://doi.org/10.1080/0952813X.2022.2092560

2022-07-10

Abstract:Speaker recognition is the task of identifying the spokesman automatically using speaker-specific features. It has been a popular and most involved topic in the field of speech technology. This field opens a wide opportunity for research and finds its application in the areas such as forensics, authentication, security, etc. In this work, a modified deep-convolutional neural network structure has been proposed for speaker identification that has improved convolution, activation, and pooling layers along with Adam's optimiser. The proposed architecture yielded the increase of prediction accuracy and reduction of Loss function when compared to the generic Convolutional Neural Network scheme. The execution of the proposed architecture is validated by various datasets and the outcomes show that the modified CNN performs better than the other state-of-the-art models regarding both accuracy (avg 99%) and loss function (avg 1%). From the analysis, it is found that the Modified-CNN suits the best for real-time speaker identification applications as the efficacy of the model does not degrade due to the effects of noise and interferences that are caused in the recording environment. Relevance of the work: Speaker Recognition is an area of interest in which ML and DL schemes, when combined, have the potential to make history in the areas of Automation and Authentication. Using a modified CNN can enhance the process by ignoring many issues such as false positives, background noise, and so on. This process can be expanded to create a Raga Identification and Therapy mechanism that can be used to treat diseases.

computer science, artificial intelligence
An Improved Speech Detection Algorithm Based on Time-domain Parameter

LIN Fan,XU Ming-Xing

DOI: https://doi.org/10.3969/j.issn.1002-137X.2006.04.047

2006-01-01

Computer Science

Abstract:This paper researches on speech detection algorithm based on time domain,and describes an adaptive,both forwards and backwards search,detecting short-term pulse noise algorithm.This algorithm uses a variety of features including the frame amplitude and zero crossing rate to calculate threshold using statistical method.And it searches much further for the unvoiced frame according to the ZCR(zero crossing rate),which is differ from unvoiced frame to background frame.This algorithm also detects impulse noise that last little.Experimental results show that this im- provement has good performance,even in noisy condition.Testing the original speech,the error rate is 5.04%,and in noisy environment with a SNR of beyond 2 dB,the rate is around 80%～90%.
Auditory model-based speech feature extraction and its application to speaker identification

Wu Xihong,Xiang Bing,Chi Huisheng

IF: 1.019

1999-01-01

Chinese Journal of Electronics

Abstract:Based on the psychological and physiological characteristics of the auditory periphery and cochlear nucleus, we constructed a novel computational auditory model to simulate them. As a result of such models, two kinds of features are extracted and presented in this paper. One is named as Weighted Average Localized Synchronized Rate Cepstrum (WALSRC), and the other is Weighted Firing Rate Cepstrum (WFRC). Both of them are applied to speaker identification. Using DTW and HMM as recognizers, good recognition accuracy is obtained under both of clean and noisy environments.
Using Phoneme Recognition and Text-Dependent Speaker Verification to Improve Speaker Segmentation for Chinese Speech.

Gang Wang,Xiaojun Wu,Thomas Fang Zheng

DOI: https://doi.org/10.21437/interspeech.2010-148

2010-01-01

Abstract:Speaker segmentation is widely used in many tasks such as multi-speaker detection and speaker tracking. The segmentation performance depends on the performance of speaker verification (SV) between two short utterances to a large extent, so the improvement of the SV performance for short utterances would give the segmentation performance a great help. In this paper, a method based on phoneme recognition and text-dependent speaker recognition is proposed. During segmentation, a phoneme sequence is first recognized using a phoneme, recognizer and then text-dependent speaker recognition based on dynamic time warping (DTW) is performed on the same phoneme in two adjacent windows. Experiments over Chinese Corpus Consortium (CCC) MSS database showed that better performance was achieved compared with the BIC method and the GLR method.

Detection and reconstruction of clipped speech for speaker recognition

Research on Speaker-Depended Isolated-Word Speech Recognition System

A Novel and Efficient Voice Activity Detector Using Shape Features of Speech Wave.

Self-attention Based Speaker Recognition Using Cluster-Range Loss

DNN-based Voice Activity Detection for Speaker Recognition

DDD: A Perceptually Superior Low-Response-Time DNN-based Declipper

Speaker Recognition Based on Pre-Trained Model and Deep Clustering

Research on truncated speech in speaker verification

Speech-Declipping Transformer with Complex Spectrogram and Learnerble Temporal Features

Weighted Cluster-Range Loss and Criticality-Enhancement Loss for Speaker Recognition

Analysis of Deep Clustering as Preprocessing for Automatic Speech Recognition of Sparsely Overlapping Speech

Speaker recognition based on improved ECAPA-TDNN network

SPEECHCLIP: INTEGRATING SPEECH WITH PRE-TRAINED VISION AND LANGUAGE MODEL

Speaker Clustering Algorithm in Speech Recognition

DNN based Speaker Recognition on Short Utterances

Efficient Identification Of Speakers In News Video Based On Shot Segmentation

Pyramid Feature Attention Network for Speech Resampling Detection

Modified layer deep convolution neural network for text-independent speaker recognition

An Improved Speech Detection Algorithm Based on Time-domain Parameter

Auditory model-based speech feature extraction and its application to speaker identification

Using Phoneme Recognition and Text-Dependent Speaker Verification to Improve Speaker Segmentation for Chinese Speech.