Abstract:Traditionally, speech quality evaluation relies on subjective assessments or intrusive methods that require reference signals or additional equipment. However, over recent years, non-intrusive speech quality assessment has emerged as a promising alternative, capturing much attention from researchers and industry professionals. This article presents a deep learning-based method that exploits large-scale intrusive simulated data to improve the accuracy and generalization of non-intrusive methods. The major contributions of this article are as follows. First, it presents a data simulation method, which generates degraded speech signals and labels their speech quality with the perceptual objective listening quality assessment (POLQA). The generated data is proven to be useful for pretraining the deep learning models. Second, it proposes to apply an adversarial speaker classifier to reduce the impact of speaker-dependent information on speech quality evaluation. Third, an autoencoder-based deep learning scheme is proposed following the principle of representation learning and adversarial training (AT) methods, which is able to transfer the knowledge learned from a large amount of simulated speech data labeled by POLQA. With the help of discriminative representations extracted from the autoencoder, the prediction model can be trained well on a relatively small amount of speech data labeled through subjective listening tests. Fourth, an end-to-end speech quality evaluation neural network is developed, which takes magnitude and phase spectral features as its inputs. This phase-aware model is more accurate than the model using only the magnitude spectral features. A large number of experiments are carried out with three datasets: one simulated with labels obtained using POLQA and two recorded with labels obtained using subjective listening tests. The results show that the presented phase-aware method improves the performance of the baseline model and the proposed model with latent representations extracted from the adversarial autoencoder (AAE) outperforms the state-of-the-art objective quality assessment methods, reducing the root mean square error (RMSE) by 10.5% and 12.2% on the Beijing Institute of Technology (BIT) dataset and Tencent Corpus, respectively. The code and supplementary materials are available at https://github.com/liushenme/AAE-SQA.

A Novel Multi-task Learning Based Automatic Speech Impairment Assessment Algorithm

Automatic Assessment of Depression from Speech Via a Hierarchical Attention Transfer Network and Attention Autoencoders

Multi-Modal Multi-Scale Speech Expression Evaluation In Computer-Assisted Language Learning

Automatic Assessment of Dysarthria Using Audio-visual Vowel Graph Attention Network

Siamese Neural Network for Speech-Based Depression Classification and Severity Assessment

Non-Intrusive Speech Quality Assessment Based on Deep Neural Networks for Speech Communication

Deep Learning-based Non-Intrusive Multi-Objective Speech Assessment Model with Cross-Domain Features

Multitask and Transfer Learning Approach for Joint Classification and Severity Estimation of Dysphonia

An AI-Based Automatic Speech Recognition for Chongqing Dialect Older Adults with Cognitive Impairment

A New Method for Predicting Severity Level of Dysarthric Speech Based on Joint Feature-Sample Selection Using Audio-Visual Data

CogniVoice: Multimodal and Multilingual Fusion Networks for Mild Cognitive Impairment Assessment from Spontaneous Speech

Speaker-Independent Dysarthria Severity Classification using Self-Supervised Transformers and Multi-Task Learning

A non-invasive speech quality evaluation algorithm for hearing aids with multi-head self-attention and audiogram-based features

Multi-task estimation of age and cognitive decline from speech

A new strategy on Early diagnosis of cognitive impairment via novel cross-lingual language markers: a non-invasive description and AI analysis for the cookie theft picture

A Multi-Objective Learning Speech Enhancement Algorithm Based on IRM Post-Processing with Joint Estimation of SCNN and TCNN

Improved Posterior Probability Estimation Methods for the Freely-Spoken Speech Evaluation

Automatic speech analysis for detecting cognitive decline of older adults

A New Benchmark of Aphasia Speech Recognition and Detection Based on E-Branchformer and Multi-task Learning

End-to-End ASR-Enhanced Neural Network for Alzheimer’s Disease Diagnosis

Deep Neural Network Based Chinese Dialect Classification