Abstract:Background Perceptual measures such as speech intelligibility are known to be biased, variant and subjective, to which an automatic approach has been seen as a more reliable alternative. On the other hand, automatic approaches tend to lack explainability, an aspect that can prevent the widespread usage of these technologies clinically. Aims In the present work, we aim to study the relationship between four perceptual parameters and speech intelligibility by automatically modelling the behaviour of six perceptual judges, in the context of head and neck cancer. From this evaluation we want to assess the different levels of relevance of each parameter as well as the different judge profiles that arise, both perceptually and automatically. Methods and Procedures Based on a passage reading task from the Carcinologic Speech Severity Index (C2SI) corpus, six expert listeners assessed the voice quality, resonance, prosody and phonemic distortions, as well as the speech intelligibility of patients treated for oral or oropharyngeal cancer. A statistical analysis and an ensemble of automatic systems, one per judge, were devised, where speech intelligibility is predicted as a function of the four aforementioned perceptual parameters of voice quality, resonance, prosody and phonemic distortions. Outcomes and Results The results suggest that we can automatically predict speech intelligibility as a function of the four aforementioned perceptual parameters, achieving a high correlation of 0.775 (Spearman's ρ). Furthermore, different judge profiles were found perceptually that were successfully modelled automatically. Conclusions and Implications The four investigated perceptual parameters influence the global rating of speech intelligibility, showing that different judge profiles emerge. The proposed automatic approach displayed a more uniform profile across all judges, displaying a more reliable, unbiased and objective prediction. The system also adds an extra layer of interpretability, since speech intelligibility is regressed as a direct function of the individual prediction of the four perceptual parameters, an improvement over more black box approaches. WHAT THIS PAPER ADDS What is already known on this subject Speech intelligibility is a clinical measure typically used in the post‐treatment assessment of speech affecting disorders, such as head and neck cancer. Their perceptual assessment is currently the main method of evaluation; however, it is known to be quite subjective since intelligibility can be seen as a combination of other perceptual parameters (voice quality, resonance, etc.). Given this, automatic approaches have been seen as a more viable alternative to the traditionally used perceptual assessments. What this study adds to existing knowledge The present work introduces a study based on the relationship between four perceptual parameters (voice quality, resonance, prosody and phonemic distortions) and speech intelligibility, by automatically modelling the behaviour of six perceptual judges. The results suggest that different judge profiles arise, both in the perceptual case as well as in the automatic models. These different profiles found showcase the different schools of thought that perceptual judges have, in comparison to the automatic judges, that display more uniform levels of relevance across all the four perceptual parameters. This aspect shows that an automatic approach promotes unbiased, reliable and more objective predictions. What are the clinical implications of this work? The automatic prediction of speech intelligibility, using a combination of four perceptual parameters, show that these approaches can achieve high correlations with the reference scores while maintaining a certain degree of explainability. The more uniform judge profiles found on the automatic case also display less biased results towards the four perceptual parameters. This aspect facilitates the clinical implementation of this class of systems, as opposed to the more subjective and harder to reproduce perceptual assessments.

On the Use of Acoustic Simulations and PESQ Measures for the Prediction of Speech Intelligibility in Sensorineural Hearing Loss

Speech intelligibility of simulated hearing loss sounds and its prediction using the Gammachirp Envelope Similarity Index (GESI)

Speech intelligibility prediction based on a physiological model of the human ear and a hierarchical spiking neural network

GESI: Gammachirp Envelope Similarity Index for Predicting Intelligibility of Simulated Hearing Loss Sounds

Outcome Prediction of Speech Perception in Quiet and in Noise for Cochlear Implant Candidates Based on Pre-Operative Measures

A Statistical Method for the Analysis of Speech Intelligibility Tests

A Data-Driven Speech Intelligibility Assessment Method Using Sum-Sorted Spectrogram Feature

An Investigation of Audibility Effects on Cochlear Implant Speech Perception Prediction

Coded Speech Quality Measurement by a Non-Intrusive PESQ-DNN

Non Intrusive Intelligibility Predictor for Hearing Impaired Individuals using Self Supervised Speech Representations

Exploiting Hidden Representations from a DNN-based Speech Recogniser for Speech Intelligibility Prediction in Hearing-impaired Listeners

Immediate improvement of speech-in-noise perception through multisensory stimulation via an auditory to tactile sensory substitution

Individual Aided Speech-Recognition Performance and Predictions of Benefit for Listeners With Impaired Hearing Employing FADE

Predicting speech perception outcomes following cochlear implantation in adults with unilateral deafness or highly asymmetric hearing loss

Non-Intrusive Speech Intelligibility Prediction for Hearing-Impaired Users using Intermediate ASR Features and Human Memory Models

Objective speech intelligibility prediction using a deep learning model with continuous speech-evoked cortical auditory responses

Non-Intrusive Binaural Speech Intelligibility Prediction from Discrete Latent Representations

Residual-Guided Non-Intrusive Speech Quality Assessment

Automatic modelling of perceptual judges in the context of head and neck cancer speech intelligibility

Non-Intrusive Speech Intelligibility Prediction for Hearing Aids using Whisper and Metadata

Objective Prediction of Hearing Aid Benefit Across Listener Groups Using Machine Learning: Speech Recognition Performance With Binaural Noise-Reduction Algorithms