Abstract:In this study, we propose a cross-domain multi-objective speech assessment model called MOSA-Net, which can estimate multiple speech assessment metrics simultaneously. Experimental results show that MOSA-Net can improve the linear correlation coefficient (LCC) by 0.026 (0.990 vs 0.964 in seen noise environments) and 0.012 (0.969 vs 0.957 in unseen noise environments) in PESQ prediction, compared to Quality-Net, an existing single-task model for PESQ prediction, and improve LCC by 0.021 (0.985 vs 0.964 in seen noise environments) and 0.047 (0.836 vs 0.789 in unseen noise environments) in STOI prediction, compared to STOI-Net (based on CRNN), an existing single-task model for STOI prediction. Moreover, MOSA-Net, originally trained to assess objective scores, can be used as a pre-trained model to be effectively adapted to an assessment model for predicting subjective quality and intelligibility scores with a limited amount of training data. Experimental results show that MOSA-Net can improve LCC by 0.018 (0.805 vs 0.787) in MOS prediction, compared to MOS-SSL, a strong single-task model for MOS prediction. In light of the confirmed prediction capability, we further adopt the latent representations of MOSA-Net to guide the speech enhancement (SE) process and derive a quality-intelligibility (QI)-aware SE (QIA-SE) approach accordingly. Experimental results show that QIA-SE provides superior enhancement performance compared with the baseline SE system in terms of objective evaluation metrics and qualitative evaluation test. For example, QIA-SE can improve PESQ by 0.301 (2.953 vs 2.652 in seen noise environments) and 0.18 (2.658 vs 2.478 in unseen noise environments) over a CNN-based baseline SE model.

A Transfer and Multi-Task Learning Based Approach for MOS Prediction

Utilizing Self-supervised Representations for MOS Prediction

Neural MOS Prediction for Synthesized Speech Using Multi-Task Learning With Spoofing Detection and Spoofing Type Classification

MOSPC: MOS Prediction Based on Pairwise Comparison

LE-SSL-MOS: Self-Supervised Learning MOS Prediction with Listener Enhancement

Deep Learning-based Non-Intrusive Multi-Objective Speech Assessment Model with Cross-Domain Features

The VoiceMOS Challenge 2023: Zero-shot Subjective Speech Quality Prediction for Multiple Domains

The T05 System for The VoiceMOS Challenge 2024: Transfer Learning from Deep Image Classifier to Naturalness MOS Prediction of High-Quality Synthetic Speech

Generalization Ability of MOS Prediction Networks

A Comparison of Deep Learning MOS Predictors for Speech Synthesis Quality

RAMP: Retrieval-Augmented MOS Prediction via Confidence-based Dynamic Weighting

Speech MOS multi-task learning and rater bias correction

DDOS: A MOS Prediction Framework utilizing Domain Adaptive Pre-training and Distribution of Opinion Scores

The VoiceMOS Challenge 2022

The VoiceMOS Challenge 2024: Beyond Speech Quality Prediction

Wavoice: an Mmwave-Assisted Noise-Resistant Speech Recognition System

Wavoice: A mmWave-assisted Noise-resistant Speech Recognition SystemJust Accepted

Wavoice: an Mmwave-Assisted Noise-Resistant Speech Recognition System.

LDNet: Unified Listener Dependent Modeling in MOS Prediction for Synthetic Speech

CCATMos: Convolutional Context-aware Transformer Network for Non-intrusive Speech Quality Assessment

Mos predictor for synthetic speech with i-vector inputs