Abstract:Abstract Recent progress in deep learning has revolutionized speech recognition research, with Deep Neural Networks (DNNs) becoming the new state of the art for acoustic modeling. DNNs offer significantly lower speech recognition error rates compared to those provided by the previously used Gaussian Mixture Models (GMMs). Unfortunately, DNNs are data sensitive, and unseen data conditions can deteriorate their performance. Acoustic distortions such as noise, reverberation, channel differences, etc. add variation to the speech signal, which in turn impact DNN acoustic model performance. A straightforward solution to this issue is training the DNN models with these types of variation, which typically provides quite impressive performance. However, anticipating such variation is not always possible; in these cases, DNN recognition performance can deteriorate quite sharply. To avoid subjecting acoustic models to such variation, robust features have traditionally been used to create an invariant representation of the acoustic space. Most commonly, robust feature-extraction strategies have explored three principal areas: (a) enhancing the speech signal, with a goal of improving the perceptual quality of speech; (b) reducing the distortion footprint, with signal-theoretic techniques used to learn the distortion characteristics and subsequently filter them out of the speech signal; and finally (c) leveraging knowledge from auditory neuroscience and psychoacoustics, by using robust features inspired by auditory perception.In this chapter, we present prominent robust feature-extraction strategies explored by the speech recognition research community, and we discuss their relevance to coping with data-mismatch problems in DNN-based acoustic modeling. We present results demonstrating the efficacy of robust features in the new paradigm of DNN acoustic models. And we discuss future directions in feature design for making speech recognition systems more robust to unseen acoustic conditions. Note that the approaches discussed in this chapter focus primarily on single channel data.

Learning An Invariant Speech Representation

An invariant convolution model and its Variational Bayesian Approximation approach via Students-t priors for acoustic imaging in colored noises

Normalized Recognition of Speech and Audio Events

Learning Invariant Representation and Risk Minimized for Unsupervised Accent Domain Adaptation

Augmentation Invariant Discrete Representation for Generative Spoken Language Modeling

Toward Domain-Invariant Speech Recognition via Large Scale Training

BYOL for Audio: Exploring Pre-Trained General-Purpose Audio Representations

A perceptual similarity space for speech based on self-supervised speech representations

Understanding Self-Supervised Learning of Speech Representation via Invariance and Redundancy Reduction

An Attribute-Aligned Strategy for Learning Speech Representation

Invariant Representations in Deep Learning for Optoacoustic Imaging

Speaker-invariant affective representation learning via adversarial training

A Generic Self-Supervised Framework of Learning Invariant Discriminative Features

Robust Features in Deep-Learning-Based Speech Recognition

Formation Of An Auditory Map For Invariant Perception Of Vowel Sounds: Listening To A Variety Of Speakers To Make Unified Vowel Representation

Orthogonality and isotropy of speaker and phonetic information in self-supervised speech representations

Speaker-Independent Acoustic-to-Articulatory Speech Inversion

Invariant Motion Representation Learning for 3D Talking Face Synthesis.

TIPAA-SSL: Text Independent Phone-to-Audio Alignment based on Self-Supervised Learning and Knowledge Transfer

Enhancing the Stability of LLM-based Speech Generation Systems through Self-Supervised Representations

Adversarial Learning of Raw Speech Features for Domain Invariant Speech Recognition