Abstract:Objective: Develop a novel and highly efficient framework that decodes Inferior Colliculus (IC) neural activities for phoneme recognition. Methods: We propose using Hyperdimensional Computing (HDC) to support an efficient phoneme recognition algorithm, in contrast to widely applied Deep Neural Networks (DNN). The high-dimensional representation and operations in HDC are rooted in human brain functionalities and naturally parallelizable, showing the potential for efficient neural activity analysis. Our proposed method includes a spatial and temporal-aware HDC encoder that effectively captures global and local patterns. As part of our framework, we deploy the lightweight HDC-based algorithm on a highly customizable and flexible hardware platform, i.e., Field Programmable Gate Arrays (FPGA), for optimal algorithm speedup. To evaluate our method, we record IC neural activities on gerbils while playing the sound of different phonemes. Results: We compare our proposed method with multiple baseline machine learning algorithms in recognition quality and learning efficiency, across different hardware platforms. The results show that our method generally achieves better classification quality than the best-performing baseline. Compared to the Deep Residual Neural Network (i.e., ResNet), our method shows a speedup up to 74×, 67×, 210× on CPU, GPU, and FPGA respectively. We achieve up to 15% (10%) higher accuracy in consonant (vowel) classification than ResNet. Conclusion: By leveraging brain-inspired HDC for IC neural activity encoding and phoneme classification, we achieve orders of magnitude runtime speedup while improving accuracy in various challenging task settings. Significance: Decoding IC neural activities is an important step to enhance understanding about human auditory system. However, these responses from the central auditory system are noisy and contain high variance, demanding large-scale datasets and iterative model fine-tuning. The proposed HDC-based framework is more scalable and viable for future real-world deployment thanks to its fast training and overall better quality.

Investigation of Tandem Deep Belief Network Approach for Phoneme Recognition

Hierarchical and Bidirectional Joint Multi-Task Classifiers for Natural Language Understanding

Tandem System Based on Competing Information

The Appropriate Hidden Layers of Deep Belief Networks for Speech Recognition

Fast learning algorithms for time-delay neural networks phoneme recognition

Effective Phoneme Decoding With Hyperbolic Neural Networks for High-Performance Speech BCIs

Bipolar Population Threshold Encoding for Audio Recognition with Deep Spiking Neural Networks

Improved Bottleneck Feature Using Hierarchical Deep Belief Networks for Keyword Spotting in Continues Speech

Phonetic Temporal Neural Model for Language Identification

Phonotactic language recognition based on DNN-HMM acoustic model

Deep joint learning for language recognition

Time-Frequency Cepstral Features and Combining Discriminative Training for Phonotactic Language Recognition

Phonetic Feature Extraction by Time-Sequence Binary Classifiers.

Time-Contrastive Learning Based Deep Bottleneck Features for Text-Dependent Speaker Verification

Tandem Multitask Training of Speaker Diarisation and Speech Recognition for Meeting Transcription

Application of Deep Belief Networks for natural language understanding

Acoustic Modeling With Dfsmn-Ctc And Joint Ctc-Ce Learning

Partial-tied-mixture Auxiliary Chain Models for Speech Recognition Based on Dynamic Bayesian Networks

Hyperdimensional Brain-Inspired Learning for Phoneme Recognition With Large-Scale Inferior Colliculus Neural Activities

Combining Information from Multi-Stream Features Using Deep Neural Network in Speech Recognition

Deep Belief Networks Based Voice Activity Detection