Abstract:In the competing speaker environments, human listeners need to focus or switch their auditory attention according to dynamic intentions. The reliable cortical tracking ability to the speech envelope is an effective feature for decoding the target speech from the neural signals. Moreover, previous studies revealed that the root mean square (RMS)–level–based speech segmentation made a great contribution to the target speech perception with the modulation of sustained auditory attention. This study further investigated the effect of the RMS-level–based speech segmentation on the auditory attention decoding (AAD) performance with both sustained and switched attention in the competing speaker auditory scenes. Objective biomarkers derived from the cortical activities were also developed to index the dynamic auditory attention states. In the current study, subjects were asked to concentrate or switch their attention between two competing speaker streams. The neural responses to the higher- and lower-RMS-level speech segments were analyzed via the linear temporal response function (TRF) before and after the attention switching from one to the other speaker stream. Furthermore, the AAD performance decoded by the unified TRF decoding model was compared to that by the speech-RMS-level–based segmented decoding model with the dynamic change of the auditory attention states. The results showed that the weight of the typical TRF component approximately 100-ms time lag was sensitive to the switching of the auditory attention. Compared to the unified AAD model, the segmented AAD model improved attention decoding performance under both the sustained and switched auditory attention modulations in a wide range of signal-to-masker ratios (SMRs). In the competing speaker scenes, the TRF weight and AAD accuracy could be used as effective indicators to detect the changes of the auditory attention. In addition, with a wide range of SMRs (i.e., from 6 to –6 dB in this study), the segmented AAD model showed the robust decoding performance even with short decision window length, suggesting that this speech-RMS-level–based model has the potential to decode dynamic attention states in the realistic auditory scenarios.

A Neural Attention Model for Disfluency Detection.

Attention-Enhanced Connectionist Temporal Classification for Discrete Speech Emotion Recognition

Semi-Supervised Disfluency Detection.

A Novel Neural Sequence Model with Multiple Attentions for Word Sense Disambiguation

Augmenting Automatic Speech Recognition Models with Disfluency Detection

DLCEncDec : A Fully Character-Level Encoder-Decoder Model for Neural Responding Conversation.

SADNet: Sustained Attention Decoding in a Driving Task by Self-Attention Convolutional Neural Network

A Speech-Level–Based Segmented Model to Decode the Dynamic Auditory Attention States in the Competing Speaker Scenes

A Recurrent Neural Model with Attention for the Recognition of Chinese Implicit Discourse Relations

D-FaST: Cognitive Signal Decoding with Disentangled Frequency-Spatial-Temporal Attention

Learning When to Attend for Neural Machine Translation

Attention-over-Attention Neural Networks for Reading Comprehension

Attention with Intention for a Neural Network Conversation Model

Encoder-decoder with focus-mechanism for sequence labelling based spoken language understanding

FluentNet: End-to-End Detection of Speech Disfluency with Deep Learning

Boosting Disfluency Detection with Large Language Model as Disfluency Generator

Neural Language Correction with Character-Based Attention

Single-word Auditory Attention Decoding Using Deep Learning Model

Decoder-Only or Encoder-Decoder? Interpreting Language Model as a Regularized Encoder-Decoder

Attention-based Belief or Disbelief Feature Extraction for Dependency Parsing.