Abstract:In our previous work, we proposed a feature compensation approach using high-order vector Taylor series (VTS) approximation for noisy speech recognition. In this paper, we report new progress on making it more powerful and practical in real applications. First, mixtures of densities are used to enhance the distortion models of both additive noise and convolutional distortion. New formulations for maximum likelihood (ML) estimation of distortion model parameters, and minimum mean squared error (MMSE) estimation of clean speech are derived and presented. Second, we improve the feature compensation in both efficiency and accuracy by applying higher order information of VTS approximation only to the noisy speech mean parameters, and a temporal smoothing operation for the posterior probability of Gaussian mixture components in clean speech estimation. Finally, we design a procedure to perform irrelevant variability normalization (IVN) based joint training of a reference Gaussian mixture model (GMM) for feature compensation and hidden Markov models (HMMs) for acoustic modeling using VTS-based feature compensation. The effectiveness of our proposed approach is confirmed by experiments on Aurora3 benchmark database for a real-world in-vehicle connected digits recognition task. Compared with ETSI advanced front-end, our approach achieves significant recognition accuracy improvement across three “training-testing” conditions for four languages.

Synthesized Stereo-Based Stochastic Mapping with Data Selection for Robust Speech Recognition.

Synthesized Stereo Mapping Via Deep Neural Networks for Noisy Speech Recognition

AudioVSR: Enhancing Video Speech Recognition with Audio Data

Stereo-based Stochastic Mapping with Context Using Probabilistic PCA for Noise Robust Automatic Speech Recognition

Stereo Hidden Markov Modeling for Noise Robust Speech Recognition

HMM-based pseudo-clean speech synthesis for splice algorithm

Wavoice: A mmWave-assisted Noise-resistant Speech Recognition SystemJust Accepted

Speech Feature Mapping based on Switching Linear Dynamic System

Robustness of HMM-based Speech Synthesis

An Approach To Robust Speaker Recognition Using Stochastic Matching

An Improved VTS Feature Compensation Using Mixture Models of Distortion and IVN Training for Noisy Speech Recognition

Voice Conversion Based on Gaussian Mixture Modules with Minimum Distance Spectral Mapping

Robust Speaker-Adaptive HMM-Based Text-to-Speech Synthesis

Discriminative Dynamic Gaussian Mixture Selection with Enhanced Robustness and Performance for Multi-Accent Speech Recognition

Improved unit selection speech synthesis method utilizing subjective evaluation results on synthetic speech

Speech Recognition with Augmented Synthesized Speech

MAP-based Speaker Adaptation in Speech Synthesis

DNN-based Stochastic Postfilter for HMM-based Speech Synthesis

A Cohort-Based Speaker Model Synthesis for Mismatched Channels in Speaker Verification

A Novel HTS System Using both Continuous HMMs and Discrete HMMs

SynesLM: A Unified Approach for Audio-visual Speech Recognition and Translation via Language Model and Synthetic Data