Abstract:Far-field multi-speaker automatic speech recognition (ASR) has drawn increasing attention in recent years. Most existing methods feature a signal processing frontend and an ASR backend. In realistic scenarios, these modules are usually trained separately or progressively, which suffers from either inter-module mismatch or a complicated training process. In this paper, we propose an end-to-end multi-channel model that jointly optimizes the speech enhancement (including speech dereverberation, denoising, and separation) frontend and the ASR backend as a single system. To the best of our knowledge, this is the first work that proposes to optimize dereverberation, beamforming, and multi-speaker ASR in a fully end-to-end manner. The frontend module consists of a weighted prediction error (WPE) based submodule for dereverberation and a neural beamformer for denoising and speech separation. For the backend, we adopt a widely used end-to-end (E2E) ASR architecture. It is worth noting that the entire model is differentiable and can be optimized in a fully end-to-end manner using only the ASR criterion, without the need of parallel signal-level labels. We evaluate the proposed model on several multi-speaker benchmark datasets, and experimental results show that the fully E2E ASR model can achieve competitive performance on both noisy and reverberant conditions, with over 30% relative word error rate (WER) reduction over the single-channel baseline systems.

Design and Implementation of End-Point Detection Accelerator for Speech Recognition

Real-time Speaker Recognition System for PDA

Implementation of Abnormal Sound Detection in Intelligent Surveillance Front-end System

A High Performance FPGA-Based Accelerator Design for End-to-End Speaker Recognition System

A Low-power and High-accuracy Accelerator with Voice Classification for Keyword Spotting

A Double Threshold Speech Endpoint Detection Algorithm Implementation in ASIC

Efficient Binary Weight Convolutional Network Accelerator for Speech Recognition

A Robust Algorithm For Real-Time Endpoint Detection In The Noisy Mobile Environments

An embedded speech recognition system solution based on acceleration modules

A Robust, Real-Time Endpoint Detector With Energy Normalization For Asr In Adverse Environments

Real-time Architecture for Audio-Visual Active Speaker Detection.

An Efficient Robust Asr System Based On The Combination Of Speech Enhancement And Hmm Adaptation

Effective Speech Endpoint Detection Algorithm For Voiceprint Recognition

Robust endpoint detection and energy normalization for real-time speech and speaker recognition

A hardware accelerator for speech recognition applications

Endpoint Detection of Chinese Digital Speech Based on Finite State Machine

94 GHz Millimeter Wave Conducted Speech Enhancement

Endpoint Detect Method of Embedded Speech Recognition System

A Robust Acoustic Echo Canceller for Noisy Environment

End-to-End Dereverberation, Beamforming, and Speech Recognition in a Cocktail Party.

Optimization of ETSI DSR frontend software on a high-efficient audio DSP