Abstract:The advantage and limitations of utilizing automatic speech recognition (ASR) techniques for modelling human speech recognition are investigated for a set of ``critical'' speech maskers for which many standard models of human speech recognition fail. A deep neural net (DNN)-based ASR system utilizing a closed-set sentence recognition test is used to model the speech recognition threshold (SRT) of normal-hearing listeners for a variety of noise types. The benchmark data from Schubotz et al. (2016) include SRTs measured in conditions with an increasing complexity in terms of spectro-temporal modulation (from stationary speech-shaped noise to a single interfering talker). The DNN-based model as proposed in Spille et al. (2018) produces a higher prediction accuracy than baseline models (i.e., SII, ESII, STOI, and mr-sESPM) even though it does not require a clean speech reference signal (as is the case for most auditory model-based SRT predictions). The most accurate predictions are obtained with multi-condition training with known noise types and ASR features that explicitly account for temporal modulations in noisy sentences. Another advantage of the approach is that the DNN can serve as valuable analysis tool to uncover signal recognition strategies: For instance, by identifying the most relevant cues for correct classification in modulated noise, it is shown that the DNN is listening in the dips. Finally, we present preliminary data indicating that the WER of the model can be replaced with an estimate of the WER, which does not require the transcript of utterances during test time and therefore eliminates an important limitation of the previous model that prevents it from being used in real-world scenarios.

Adaptive Reverberation Absorption using Non-stationary Masking Components Detection for Intelligibility Improvement

Cross-modal Mask Fusion and Modality-Balanced Audio-Visual Speech Recognition

Harmonic and non-Harmonic Based Noisy Reverberant Speech Enhancement in Time Domain

An Improved Metric of Informational Masking for Perceptual Audio Quality Measurement

Sub-band Adaptive Noise Reduction Algorithm to Improve Speech Intelligibility

Millimeter Wave Conduct Speech Enhancement Based on Auditory Masking Properties

A Speech Enhancement Algorithm Based On Non-Linear Filtering And Noise Masking

Modelling human speech recognition in challenging noise maskers using machine learning

Improvement of intelligibility of ideal binary-masked noisy speech by adding background noise.

Phoneme-Based Ratio Mask Estimation for Reverberant Speech Enhancement in Cochlear Implant Processors

The Ability of Temporally Integrating Acoustic Waveforms is Associated with Release of Speech from Informational Masking under Reverberant Conditions

Impact of Mask Type as Training Target for Speech Intelligibility and Quality in Cochlear-Implant Noise Reduction

Non-intrusive intelligibility prediction for Mandarin speech in noise

Effect of enhancement of spectral changes on speech intelligibility and clarity preferences for the hearing impaired.

Attentional Modulation of the Early Cortical Representation of Speech Signals in Informational or Energetic Masking

A New Observation Model in the Logarithmic Mel Power Spectral Domain for the Automatic Recognition of Noisy Reverberant Speech

A generalized time-frequency subtraction method for robust speech enhancement based on wavelet filter banks modeling of human auditory system

Assessing Level-Dependent Segmental Contribution to the Intelligibility of Speech Processed by Single-Channel Noise-Suppression Algorithms

Multi-channel adaptive dereverberation robust to abrupt change of target speaker position.

Title Non-intrusive intelligibility prediction for Mandarin speech innoise

Using an Adjustment Training and a Smoothing Mask for Speech Segregation