Abstract:As automatic speech recognition evolves, deployment of the voice user interface (VUI) has boomingly expanded. Especially since the COVID-19 pandemic, the VUI has gained more attention in online communication owing to its non-contact property. However, the VUI struggles to be applied in public scenes due to the degradation of received audio signals caused by various ambient noises. In this article, we propose Wavoice , the first noise-resistant multi-modal speech recognition system that fuses two distinct voices sensing modalities (i.e., millimeter-wave signals and audio signals from a microphone) together. One key contribution is to model the inherent correlation between millimeter-wave and audio signals. Based on it, Wavoice facilitates the real-time noise-resistant voice activity detection and user targeting from multiple speakers. Additionally, we elaborate on two novel modules for multi-modal fusion embedded into the neural network, leading to accurate speech recognition. Extensive experiments prove the effectiveness of Wavoice under adverse conditions—that is, the character recognition error rate below 1% in a range of 7 m. In terms of robustness and accuracy, Wavoice considerably outperforms existing audio-only speech recognition methods with lower character error and word error rates.

Speaker recognition based on VQMAP model and AdaBoost learning algorithm

Agmma: A Novel Incremental Adaptation Method And Its Application To Speaker Recognition

A Novel and Efficient Voice Activity Detector Using Shape Features of Speech Wave.

Real-time Speaker Recognition System for PDA

Accelerate Training By Dwt In Speaker Identification Using Svms

Maximum Likelihood I-Vector Space Using PCA for Speaker Verification.

Learning Virtual HD Model for Bi-model Emotional Speaker Recognition

Preliminary Study on Self-contained UBM Construction for Speaker Recognition.

The Improved VQ Algorithm for Speaker Recognition

Ensemble of Support Vector Machine for Text-Independent Speaker Recognition

Wavoice: A mmWave-assisted Noise-resistant Speech Recognition SystemJust Accepted

Noise Robust Speaker Recognition Based on Adaptive Frame Weighting in GMM for i-Vector Extraction.

An algorithm for efficient speaker identification using reference speaker model based two-layer structure

Speaker Identification Using the VQ-Based Discriminative Kernels

Using MCE Algorithm to Improve the Performance of Speaker Recognition

A Speech Recognition System Based on a Hybrid HMM/SVM Architecture

Wavoice: an Mmwave-Assisted Noise-Resistant Speech Recognition System

Dynamic Speaker Selected Training for Rapid Speaker Adaptation

Speaker recognition using continuous density support vector machines

DSARSR: Deep Stacked Auto-encoders Enhanced Robust Speaker Recognition

VB-HMM Speaker Diarization with Enhanced and Refined Segment Representation.