Abstract:As automatic speech recognition evolves, the deployment of voice user interface has boomingly expanded. Especially since the COVID-19 pandemic, VUI has gained more attention in online communication owing to its non-contact property. However, VUI struggles to be applied in public scenes due to the degradation of received audio signals caused by various ambient noises. In this paper, we propose Wavoice , the first noise-resistant multi-modal speech recognition system that fuses two distinct voice sensing modalities, i.e., millimeter-wave (mmWave) signals and audio signals from a microphone, together. One key contribution is to model the inherent correlation between mmWave and audio signals. Based on it, Wavoice facilitates the real-time noise-resistant voice activity detection and user targeting from multiple speakers. Additionally, we elaborate on two novel modules for multi-modal fusion embedded into the neural network, leading to accurate speech recognition. Extensive experiments prove the effectiveness of Wavoice under adverse conditions, that is, the character recognition error rate below 1 \(\% \) in a range of 7 meters. In terms of robustness and accuracy, Wavoice considerably outperforms existing audio-only speech recognition methods with lower character error rate and word error rate.

An Anti-noise Algorithm for Speaker-Dependent Mandarin Vocal Numbers Recognition

Speech Recognition System Based on SCHMM/ANN in Noisy Environment

Mandarin Digital Speech Recognition Based on a Chaotic Neural Network and Fuzzy C-means Clustering

A Novel Recognition Method For Electronic Nose Using Artificial Neural Network And Fuzzy Recognition

Speech Recognition System Based on CDHMM/SOFMNN in Noisy Environment

The Optimal Selecting for HMM State-number in Mandarin Continuous Speech

Speech Recognition Algorithm Based on Neural Network and Hidden Markov Model

Robust F0 Modeling for Mandarin Speech Recognition in Noise.

Wavoice: A mmWave-assisted Noise-resistant Speech Recognition SystemJust Accepted

A Study of Speech Emotion Recognition Based on Hybrid Algorithm

Multi-layer Perceptron for Isolated Mandarin Vocal Number Recognition

Chinese Vowels Recognition Method Based on Non-Homogeneous Hidden Markov Model

Phonotactic language recognition based on DNN-HMM acoustic model

Noisy speech recognition performance of discriminative HMMs

Algorithm for Mandarin Continuous Speech Recognition Based on Context-Dependent Unit Between Syllables

Autoregressive Model-Based Robust Speech Recognition in Additive Noise Environment

Stereo Hidden Markov Modeling for Noise Robust Speech Recognition

New Neural Network Architecture with Application in Mandarin Digit Speech Recognition

A Noise Robust Front End Algorithm for Mandarin Speech Recognition and Performance Analysis

Mandarin Continuous Digit Speech Recognition System

High Performance Digit Mandarin Speech Recognition