Abstract:In this paper, a new two-step joint optimization approach based on the asynchronous subregion optimization method is proposed for training a pipeline model composed of two different models. The first-step processing of the proposed joint optimization approach trains the front-end model only, and the second-step processing trains all the parameters of the combined model together. In the asynchronous subregion optimization method, the first-step processing only supports the goal of the front-end model. However, the first-step processing of the proposed approach works with a new loss function to make the front-end model support the goal of the back-end model. The proposed optimization approach was applied, here, to a pipeline composed of a deep complex convolutional recurrent network (DCCRN)-based speech enhancement model and a conformer-transducer-based ASR model as a front-end and a back-end, respectively. Then, the performance of the proposed two-step joint optimization approach was evaluated on the LibriSpeech automatic speech recognition (ASR) corpus in noisy environments by measuring the character error rate (CER) and word error rate (WER). In addition, an ablation study was carried out to examine the effectiveness of the proposed optimization approach on each of the processing blocks in the conformer-transducer ASR model. Consequently, it was shown from the ablation study that the conformer-transducer-based ASR model with the joint network trained only by the proposed optimization approach achieved the lowest average CER and WER. Moreover, the proposed optimization approach reduced the average CER and WER on the Test-Noisy dataset under matched noise conditions by 0.30% and 0.48%, respectively, compared to the approach of separate optimization of speech enhancement and ASR. Compared to the conventional two-step joint optimization approach, the proposed optimization approach provided average CER and WER reductions of 0.22% and 0.31%, respectively. Moreover, it was revealed that the proposed optimization approach achieved a lower average CER and WER, by 0.32% and 0.43%, respectively, than the conventional optimization approach under mismatched noise conditions.

Joint compensation of noise and channel in speech recognition

An Algorithm of Model Compensation Based on the Estimation of Additive Noise and Channel Function for Speech Recognition

Supervised Monaural Speech Enhancement Using Two-Level Complementary Joint Sparse Representations

An efficient joint training model for monaural noisy-reverberant speech recognition

Compensation of Speech Enhancement Distortion for Robust Speech Recognition

Residual Noise Compensation For Robust Speech Recognition In Nonstationary Noise

Channel Compensation Technique HNSSM for Speaker Recognition

Channel Adaptation Method Based on Spectral Adjusting for Speech Recognition

speech and noise dual-stream spectrogram refine network with speech distortion loss for robust speech recognition

Log-Add Approximation On-Line Noise Compensation Method Adopting Robust Decision Rules for Robust Digits Recognition

Two-Step Joint Optimization with Auxiliary Loss Function for Noise-Robust Speech Recognition

Joint Training for Simultaneous Speech Denoising and Dereverberation with Deep Embedding Representations

Joint sparse representation based cepstral-domain dereverberation for distant-talking speech recognition

Monaural Speech Enhancement Using Joint Dictionary Learning with Cross-Coherence Penalties

A Comparative Study of Noise Estimation Algorithms for Nonlinear Compensation in Robust Speech Recognition

MAP-based Audio Coding Compensation for Speaker Recognition.

Joint Training of Speech Enhancement and Self-supervised Model for Noise-robust ASR

Gaussian Specific Compensation for Channel Distortion in Speech Recognition

Channel Compensation for Robust Telephone Speech Recognition

Enhancing Anti-spoofing Countermeasures Robustness through Joint Optimization and Transfer Learning