Abstract:In this study, we propose a cross-domain multi-objective speech assessment model called MOSA-Net, which can estimate multiple speech assessment metrics simultaneously. Experimental results show that MOSA-Net can improve the linear correlation coefficient (LCC) by 0.026 (0.990 vs 0.964 in seen noise environments) and 0.012 (0.969 vs 0.957 in unseen noise environments) in PESQ prediction, compared to Quality-Net, an existing single-task model for PESQ prediction, and improve LCC by 0.021 (0.985 vs 0.964 in seen noise environments) and 0.047 (0.836 vs 0.789 in unseen noise environments) in STOI prediction, compared to STOI-Net (based on CRNN), an existing single-task model for STOI prediction. Moreover, MOSA-Net, originally trained to assess objective scores, can be used as a pre-trained model to be effectively adapted to an assessment model for predicting subjective quality and intelligibility scores with a limited amount of training data. Experimental results show that MOSA-Net can improve LCC by 0.018 (0.805 vs 0.787) in MOS prediction, compared to MOS-SSL, a strong single-task model for MOS prediction. In light of the confirmed prediction capability, we further adopt the latent representations of MOSA-Net to guide the speech enhancement (SE) process and derive a quality-intelligibility (QI)-aware SE (QIA-SE) approach accordingly. Experimental results show that QIA-SE provides superior enhancement performance compared with the baseline SE system in terms of objective evaluation metrics and qualitative evaluation test. For example, QIA-SE can improve PESQ by 0.301 (2.953 vs 2.652 in seen noise environments) and 0.18 (2.658 vs 2.478 in unseen noise environments) over a CNN-based baseline SE model.

LDNet: Unified Listener Dependent Modeling in MOS Prediction for Synthetic Speech

Generalization Ability of MOS Prediction Networks

MooseNet: A Trainable Metric for Synthesized Speech with a PLDA Module

MBNET: MOS PREDICTION FOR SYNTHESIZED SPEECH WITH MEAN-BIAS NETWORK

Deep Learning-based Non-Intrusive Multi-Objective Speech Assessment Model with Cross-Domain Features

LE-SSL-MOS: Self-Supervised Learning MOS Prediction with Listener Enhancement

A Transfer and Multi-Task Learning Based Approach for MOS Prediction

Utilizing Self-supervised Representations for MOS Prediction

The VoiceMOS Challenge 2023: Zero-shot Subjective Speech Quality Prediction for Multiple Domains

Neural MOS Prediction for Synthesized Speech Using Multi-Task Learning With Spoofing Detection and Spoofing Type Classification

A Comparison of Deep Learning MOS Predictors for Speech Synthesis Quality

The VoiceMOS Challenge 2022

DDOS: A MOS Prediction Framework utilizing Domain Adaptive Pre-training and Distribution of Opinion Scores

Partial Rank Similarity Minimization Method for Quality MOS Prediction of Unseen Speech Synthesis Systems in Zero-Shot and Semi-supervised setting

MOSPC: MOS Prediction Based on Pairwise Comparison

MTI-Net: A Multi-Target Speech Intelligibility Prediction Model

Improving LPCNet-based Text-to-Speech with Linear Prediction-structured Mixture Density Network

Speech MOS multi-task learning and rater bias correction

MOS-FAD: Improving Fake Audio Detection Via Automatic Mean Opinion Score Prediction

Attention-Based Speech Enhancement Using Human Quality Perception Modeling

The VoiceMOS Challenge 2024: Beyond Speech Quality Prediction