Abstract:Speech brain-computer interfaces (BCIs), which translate brain signals into spoken words or sentences, have shown significant potential for high-performance BCI communication. Phonemes are the fundamental units of pronunciation in most languages. While existing speech BCIs have largely focused on English, where words contain diverse compositions of phonemes, Chinese Mandarin is a monosyllabic language, with words typically consisting of a consonant and a vowel. This feature makes it feasible to develop high-performance Mandarin speech BCIs by decoding phonemes directly from neural signals. This study aimed to decode spoken Mandarin phonemes using intracortical neural signals. We observed that phonemes with similar pronunciations were often represented by inseparable neural patterns, leading to confusion in phoneme decoding. This finding suggests that the neural representation of spoken phonemes has a hierarchical structure. To account for this, we proposed learning the neural representation of phoneme pronunciation in a hyperbolic space, where the hierarchical structure could be more naturally optimized. Experiments with intracortical neural signals from a Chinese participant showed that the proposed model learned discriminative and interpretable hierarchical phoneme representations from neural signals, significantly improving Chinese phoneme decoding performance and achieving state-of-the-art. The findings demonstrate the feasibility of constructing high-performance Chinese speech BCIs based on phoneme decoding.

A k-climax neighbors policy based viterbi decoding for large vocabulary mandarin speech recognition

A Hierarchical Viterbi Algorithm For Mandarin Hybrid Speech Synthesis System

Construction of a compact dynamic decoder network for large vocabulary continuous speech recognition

Algorithm for Mandarin Continuous Speech Recognition Based on Context-Dependent Unit Between Syllables

Research on Inter-Syllable Context-Dependent Acoustic Unit for Mandarin Continuous Speech Recognition.

Duration Model for post-processing in a Mandarin speech recognition system

Effective Phoneme Decoding With Hyperbolic Neural Networks for High-Performance Speech BCIs

A Novel Efficient Decoding Algorithm for CDHMM-based Speech Recognizer on Chip.

Research on Context-Dependent Acoustical Unit (Triphone) for Mandarin Continuous Speech Recognition

Multi-Level Modeling Units for End-to-End Mandarin Speech Recognition

A comparative study on selecting acoustic modeling units in deep neural networks based large vocabulary Chinese speech recognition

A Cluster-Based Multiple Deep Neural Networks Method for Large Vocabulary Continuous Speech Recognition

An efficient viterbi algorithm on DBNs

Reliable accent specific unit generation with dynamic Gaussian mixture selection for multi-accent speech recognition

Replacing Uncertainty Decoding with Subband Re-Estimation for Large Vocabulary Speech Recognition in Noise.

Improving deep neural networks for LVCSR using dropout and shrinking structure

Parametric model of introducing inter-frame correlation information into hidden markov model for speech recognition

An Efficient Layer-Wised Beam Pruning Algorithm for Large Vocabulary Continuous Speech Recognition System

Nonlinear Regularization Decoding Method for Speech Recognition

A comparable study of modeling units for end-to-end Mandarin speech recognition

Decoding Chinese phonemes from intracortical brain signals with hyperbolic-space neural representations