Abstract:Speech brain-computer interface (speech BCI) aims to reconstruct speech from recorded brain signals. Real-time speech BCI relies on speech detection, which is greatly impacted by the selection of speech-related neural frequency features. However, most studies did not investigate this aspect when designing speech detectors. In this study, both electrocorticography (ECoG) dataset and stereo-electroencephalography (sEEG) dataset were utilized to investigate the impact of brain signal type on the contribution of frequency bands to speech detection. We calculated the mutual information (MI) between neural frequency bands and the audio envelope and found that the distributions of frequency bands varied between the two types of brain signals. Specifically, the 40-60Hz of ECoG signal and 0-20Hz of sEEG signal got the highest MI values. To address this, we propose a two-module detector that combines convolutional neural networks and long short-term memory (CNN-LSTM) for feature extraction and speech prediction. Our detector outperformed three commonly used detectors, including Linear discriminant analysis (LDA), Support Vector Machine (SVM), and LSTM. Notably, a high correlation was found between CNN output and the frequency bands, and high MI values were observed in both types of brain signals. These findings confirm the interpretability and generalizability of our proposed speech detector.

Robust Speech Detector for Speech Recognition Applications

A Speech Detection Algorithm for Speech Recognition Systems on Chip

Reliability detection by Fuzzy SVM with UBM Component feature for emotional speaker recognition

A Novel and Efficient Voice Activity Detector Using Shape Features of Speech Wave.

Endpoint Detection of Chinese Digital Speech Based on Finite State Machine

Robust speech/non-speech detection in adverse conditions using the fuzzy polarity correlation method

Speech Emotion Recognition Based on Linear Discriminant Analysis and Support Vector Machine Decision Tree

An Interpretable and Generalizable Speech Detector Based on a CNN-LSTM Framework

Real-time Speech Emotion Recognition Based on Syllable-Level Feature Extraction

Robust Speech And Non-Speech Detection

Real-time Architecture for Audio-Visual Active Speaker Detection.

Wavoice: A mmWave-assisted Noise-resistant Speech Recognition SystemJust Accepted

Speech Emotion Recognition Based on Feature Selection and Extreme Learning Machine Decision Tree

Speech Emotion Recognition Based on Syllable-Level Feature Extraction

Robust Speaker Recognition against Synthetic Speech

Real-Time Speech Recognition Method for Embedded System

A Robust Algorithm For Real-Time Endpoint Detection In The Noisy Mobile Environments

An Auditory System-Based Feature for Robust Speech Recognition.

Robust Speech Detection with Heteroscedastic Discriminant Analysis Applied to the Time-frequency Energy

Robust Spoof Speech Detection Based on Multi-Scale Feature Aggregation and Dynamic Convolution.

A High-Performance Auditory Feature for Robust Speech Recognition