Super-Arc-Connected and Super-Connected Total Digraphs

Juan Liu,J. Meng

Abstract:

What problem does this paper attempt to address?

Decoding auditory attention (in real time) with eeg

Edmund Lalor,Nima Mesgarani,Siddharth Rajaram,Adam O'Donovan,James Wright,Inyong Choi,Jonathan Brumberg,Nai Ding,Adrian KC Lee,Nils Peters,Sudarshan Ramenahalli,Jeffrey Pompe,Barbara Shinn-Cunningham,Malcolm Slaney,Shihab Shamma

2013-01-01

Abstract:BackgroundBoth magnetoencephalography and electrocorticography recordings have been used to decode which of two competing sources a listener is attending. However, it is not clear whether these techniques might work with Electroencephalography (EEG), particularly in a real-time system. We therefore set out to decode a listener’s attentional focus from EEG signals in real time, knowledge that could be incorporated into next-generation assistive listening devices.
Riemannian geometry-based decoding of the directional focus of auditory attention using EEG

Simon Geirnaert,Tom Francart,Alexander Bertrand

DOI: https://doi.org/10.48550/arXiv.2010.07171

2021-02-01

Abstract:Auditory attention decoding (AAD) algorithms decode the auditory attention from electroencephalography (EEG) signals that capture the listener's neural activity. Such AAD methods are believed to be an important ingredient towards so-called neuro-steered assistive hearing devices. For example, traditional AAD decoders allow detecting to which of multiple speakers a listener is attending to by reconstructing the amplitude envelope of the attended speech signal from the EEG signals. Recently, an alternative paradigm to this stimulus reconstruction approach was proposed, in which the directional focus of auditory attention is determined instead, solely based on the EEG, using common spatial pattern filters (CSP). Here, we propose Riemannian geometry-based classification (RGC) as an alternative for this CSP approach, in which the covariance matrix of a new EEG segment is directly classified while taking its Riemannian structure into account. While the proposed RGC method performs similarly to the CSP method for short decision lengths (i.e., the amount of EEG samples used to make a decision), we show that it significantly outperforms it for longer decision window lengths.

Signal Processing
What are we really decoding? Unveiling biases in EEG-based decoding of the spatial focus of auditory attention

Iustina Rotaru,Simon Geirnaert,Nicolas Heintz,Iris Van de Ryck,Alexander Bertrand,Tom Francart

DOI: https://doi.org/10.1088/1741-2552/ad2214

IF: 5.043

2024-01-25

Journal of Neural Engineering

Abstract:Objective. Spatial auditory attention decoding (Sp-AAD) refers to the task of identifying the direction of the speaker to which a person is attending in a multi-talker setting, based on the listener's neural recordings, e.g., electroencephalography (EEG). The goal of this study is to thoroughly investigate potential biases when training such Sp-AAD decoders on EEG data, particularly eye-gaze biases and latent trial-dependent confounds, which may result in Sp-AAD models that decode eye-gaze or trial-specific fingerprints rather than spatial auditory attention. Approach. We designed a two-speaker audiovisual Sp-AAD protocol in which the spatial auditory and visual attention were enforced to be either congruent or incongruent, and we recorded EEG data from sixteen participants undergoing several trials recorded at distinct timepoints. We trained a simple linear model for Sp-AAD based on common spatial patterns (CSP) filters in combination with either linear discriminant analysis (LDA) or k-means clustering, and evaluated them both across- and within-trial. Main results. We found that even a simple linear Sp-AAD model is susceptible to overfitting to confounding signal patterns such as eye-gaze and trial fingerprints (e.g., due to feature shifts across trials), resulting in artificially high decoding accuracies. Furthermore, we found that changes in the EEG signal statistics across trials deteriorate the trial generalization of the classifier, even when the latter is retrained on the test trial with an unsupervised algorithm. Significance. Collectively, our findings confirm that there exist subtle biases and confounds that can strongly interfere with the decoding of spatial auditory attention from EEG. It is expected that more complicated non-linear models based on deep neural networks, which are often used for Sp-AAD, are even more vulnerable to such biases. Future work should perform experiments and model evaluations that avoid and/or control for such biases in Sp-AAD tasks.

engineering, biomedical,neurosciences
Low Latency Auditory Attention Detection with Common Spatial Pattern Analysis of EEG Signals.

Siqi Cai,Enze Su,Yonghao Song,Longhan Xie,Haizhou Li

DOI: https://doi.org/10.21437/interspeech.2020-2496

2020-01-01

Abstract:A listener listens to one speech stream at a time in a multispeaker scenario. EEG-based auditory attention detection (AAD) aims to identify to which speech stream the listener has attended using EEG signals. The performance of linear modeling approaches is limited due to the non-linear nature of the human auditory perception. Furthermore, the real-world applications call for low latency AAD solutions in noisy environments. In this paper, we propose to adopt common spatial pattern (CSP) analysis to enhance the discriminative ability of EEG signals. We study the use of convolutional neural network (CNN) as the non-linear solution. The experiments show that it is possible to decode auditory attention within 2 seconds, with a competitive accuracy of 80.2%, even in noisy acoustic environments. The results are encouraging for brain-computer interfaces, such as hearing aids, which require real-time responses, and robust AAD in complex acoustic environments.
Using Ear-EEG to Decode Auditory Attention in Multiple-speaker Environment

Haolin Zhu,Yujie Yan,Xiran Xu,Zhongshu Ge,Pei Tian,Xihong Wu,Jing Chen

2024-09-13

Abstract:Auditory Attention Decoding (AAD) can help to determine the identity of the attended speaker during an auditory selective attention task, by analyzing and processing measurements of electroencephalography (EEG) data. Most studies on AAD are based on scalp-EEG signals in two-speaker scenarios, which are far from real application. Ear-EEG has recently gained significant attention due to its motion tolerance and invisibility during data acquisition, making it easy to incorporate with other devices for applications. In this work, participants selectively attended to one of the four spatially separated speakers' speech in an anechoic room. The EEG data were concurrently collected from a scalp-EEG system and an ear-EEG system (cEEGrids). Temporal response functions (TRFs) and stimulus reconstruction (SR) were utilized using ear-EEG data. Results showed that the attended speech TRFs were stronger than each unattended speech and decoding accuracy was 41.3\% in the 60s (chance level of 25\%). To further investigate the impact of electrode placement and quantity, SR was utilized in both scalp-EEG and ear-EEG, revealing that while the number of electrodes had a minor effect, their positioning had a significant influence on the decoding accuracy. One kind of auditory spatial attention detection (ASAD) method, STAnet, was testified with this ear-EEG database, resulting in 93.1% in 1-second decoding window. The implementation code and database for our work are available on GitHub: <a class="link-external link-https" href="https://github.com/zhl486/Ear_EEG_code.git" rel="external noopener nofollow">this https URL</a> and Zenodo: <a class="link-external link-https" href="https://zenodo.org/records/10803261" rel="external noopener nofollow">this https URL</a>.

Signal Processing,Sound,Audio and Speech Processing
Comparison of linear and nonlinear methods for decoding selective attention to speech from ear-EEG recordings

Mike Thornton,Danilo Mandic,Tobias Reichenbach

2024-11-15

Abstract:Many people with hearing loss struggle to comprehend speech in crowded auditory scenes, even when they are using hearing aids. It has recently been demonstrated that the focus of a listener's selective attention to speech can be decoded from their electroencephalography (EEG) recordings, raising the prospect of smart EEG-steered hearing aids which restore speech comprehension in adverse acoustic environments (such as the cocktail party). To this end, we here assess the feasibility of using a novel, ultra-wearable ear-EEG device to classify the selective attention of normal-hearing listeners who participated in a two-talker competing-speakers experiment. Eighteen participants took part in a diotic listening task, whereby they were asked to attend to one narrator whilst ignoring the other. Encoding models were estimated from the recorded signals, and these confirmed that the device has the ability to capture auditory responses that are consistent with those reported in high-density EEG studies. Several state-of-the-art auditory attention decoding algorithms were next compared, including stimulus-reconstruction algorithms based on linear regression as well as non-linear deep neural networks, and canonical correlation analysis (CCA). Meaningful markers of selective auditory attention could be extracted from the ear-EEG signals of all 18 participants, even when those markers were derived from relatively short EEG segments of just five seconds in duration. Algorithms which related the EEG signals to the rising edges of the speech temporal envelope (onset envelope) were more successful than those which made use of the temporal envelope itself. The CCA algorithm achieved the highest mean attention decoding accuracy, although differences between the performances of the three algorithms were both small and not statistically significant when EEG segments of short durations were employed.

Audio and Speech Processing
Electroencephalogram-based Multiclass Auditory Attention Decoding of Attended Speaker Direction

Yuanming Zhang,Jing Lu,Zhibin Lin,Fei Chen,Haoliang Du,Xia Gao

DOI: https://doi.org/10.36227/techrxiv.24592356.v1

2023-01-01

Abstract:Decoding the directional focus of an attended speaker from listeners’ electroencephalogram signals is an important part of a practical brain-computer interface device aimed at improving the quality of life for individuals with hearing impairment. Existing works focus on binary directional focus decoding, i.e., determining whether the attended speaker is on the left or right side of the listener. However, the information brought by the binary decoding to the subsequent speech processing algorithm is limited in practical applications, and more precise decoding of the exact direction of the attended speaker is desired. In this paper, we first present a new dataset with 15 alternative speaker directions, and then demonstrate the feasibility of multiclass directional focus decoding of attended speakers by applying our recently proposed learnable spatial mapping (LSM) module, the benefit of which has already been proven in binary decoding scenarios. Apart from combining the LSM module with the convolutional neural network (CNN), we further validate its benefit by combining it with the spectro-spatial-temporal convolutional recurrent network (CRN), a recently proposed state-of-the-art model for binary directional focus decoding. The proposed LSM-CNN and LSM-CRN models achieve a noteworthy decoding accuracy of 85.7% and 87.5%, respectively, in the presented subject-independent dataset with a decision window length of 1 second. Comprehensive experiments not only substantiate the advantages attributed to the LSM module, but also verify the influence of the decision window length, the distraction caused by interfering speakers, and the contribution of different subband of EEG signals.
A DenseNet-based method for decoding auditory spatial attention with EEG

Xiran Xu,Bo Wang,Yujie Yan,Xihong Wu,Jing Chen

2024-01-17

Abstract:Auditory spatial attention detection (ASAD) aims to decode the attended spatial location with EEG in a multiple-speaker setting. ASAD methods are inspired by the brain lateralization of cortical neural responses during the processing of auditory spatial attention, and show promising performance for the task of auditory attention decoding (AAD) with neural recordings. In the previous ASAD methods, the spatial distribution of EEG electrodes is not fully exploited, which may limit the performance of these methods. In the present work, by transforming the original EEG channels into a two-dimensional (2D) spatial topological map, the EEG data is transformed into a three-dimensional (3D) arrangement containing spatial-temporal information. And then a 3D deep convolutional neural network (DenseNet-3D) is used to extract temporal and spatial features of the neural representation for the attended locations. The results show that the proposed method achieves higher decoding accuracy than the state-of-the-art (SOTA) method (94.3% compared to XANet's 90.6%) with 1-second decision window for the widely used KULeuven (KUL) dataset, and the code to implement our work is available on Github: <a class="link-external link-https" href="https://github.com/xuxiran/ASAD_DenseNet" rel="external noopener nofollow">this https URL</a>

Signal Processing,Machine Learning
Deep learning-based auditory attention decoding in listeners with hearing impairment

M Asjid Tanveer,Martin A Skoglund,Bo Bernhardsson,Emina Alickovic

DOI: https://doi.org/10.1088/1741-2552/ad49d7

IF: 5.043

2024-05-12

Journal of Neural Engineering

Abstract:Objective: This study develops a deep learning method for fast auditory attention decoding (AAD) using electroencephalography (EEG) from listeners with hearing impairment. It addresses three classification tasks: differentiating noise from speech-in-noise, classifying the direction of attended speech (left vs. right) and identifying the activation status of hearing aid noise reduction (NR) algorithms (OFF vs. ON). These tasks contribute to our understanding of how hearing technology influences auditory processing in the hearing-impaired population. Method: Deep convolutional neural network (DCNN) models were designed for each task. Two training strategies were employed to clarify the impact of data splitting on AAD tasks: inter-trial, where the testing set used classification windows from trials that the training set hadn't seen, and intra-trial, where the testing set used unseen classification windows from trials where other segments were seen during training. The models were evaluated on EEG data from 31 participants with hearing impairment, listening to competing talkers amidst background noise. Results: Using 1-second classification windows, DCNN models achieve accuracy (ACC) of 69.8\%, 73.3\% and 82.9\% and area-under-curve (AUC) of 77.2\%, 80.6\% and 92.1\% for the three tasks respectively on inter-trial strategy. In the intra-trial strategy, they achieved ACC of 87.9\%, 80.1\% and 97.5\%, along with AUC of 94.6\%, 89.1\%, and 99.8\%. Our DCNN models show good performance on short 1-second EEG samples, making them suitable for real-world applications. Conclusion: Our DCNN models successfully addressed three tasks with short 1-second EEG windows from participants with hearing impairment, showcasing their potential. While the inter-trial strategy demonstrated promise for assessing AAD, the intra-trial approach yielded inflated results, underscoring the important role of proper data splitting in EEG-based AAD tasks. Significance: Our findings showcase the promising potential of EEG-based tools for assessing auditory attention in clinical contexts and advancing hearing technology, while also promoting further exploration of alternative deep learning architectures and their potential constraints.

engineering, biomedical,neurosciences
Auditory attention decoding from electroencephalography based on long short-term memory networks

Yun Lu,Mingjiang Wang,Longxin Yao,Hongcai Shen,Wanqing Wu,Qiquan Zhang,Lu Zhang,Moran Chen,Hao Liu,Rongchao Peng,Ming Liu,Shixiong Chen

DOI: https://doi.org/10.1016/j.bspc.2021.102966

IF: 5.1

2021-09-01

Biomedical Signal Processing and Control

Abstract:<p>Available researches have confirmed that the target speech stream of human listener's selective attention in a multi-speaker scenario can be decoded from ongoing electroencephalography (EEG) signals. Due to its great potential application for the neuro-steered hearing devices, auditory attention decoding has recently received increasing attention. In practice, development of high-accuracy decoding performance and use of EEG signals with as short duration as possible are urgently needed for auditory attention decoding in real-time scenarios. However, these are still great challenges for the existing decoding methods to achieve such performances. In this paper, we propose a nonlinear modeling approach between EEG signals and speech envelopes by exploiting long short-term memory networks (LSTM), a special structure of recurrent neural networks (RNN), to decipher selective auditory attention from single-trial EEG signals, which is capable of learning long-term dependencies between EEG responses and continuous auditory stimuli. Unlike the existing linear modeling approaches, the LSTM RNN-based auditory attention decoding method is an end-to-end nonlinear decoding framework that subsumes the explicit similarity determination as part of feature learning in deep neural networks. The proposed decoding method is evaluated using the real EEG data collected from 21 subjects. Experimental results show that our proposed decoding method can achieve the state-of-the-art performance to identify the target speech in a competing two-talker scenario. When used the EEG signals with 1 s duration, the average accuracy for the 21 subjects' attention decoding is reached to 96.12%. The results show great potentials for the real-time auditory attention detection in neuro-steered hearing aids.</p>

engineering, biomedical
Real-time control of a hearing instrument with EEG-based attention decoding

Jens Hjortkjaer,Daniel D.E. Wong,Alessandro Catania,Jonatan Marcher-Rorsted,Enea Ceolini,Soren Asp Fuglsang,Ilya Kiselev,Giovanni Di Liberto,Shih-Chii Liu,Torsten Dau,Malcolm Slaney,Alain de Cheveigne

DOI: https://doi.org/10.1101/2024.03.01.582668

2024-06-12

Abstract:Enhancing speech perception in everyday noisy acoustic environments remains an outstanding challenge for hearing aids. Speech separation technology is improving rapidly, but hearing devices cannot fully exploit this advance without knowing which sound sources the user wants to hear. Even with high-quality source separation, the hearing aid must know which speech streams to enhance and which to suppress. Advances in EEG-based decoding of auditory attention raise the potential of neurosteering, in which a hearing instrument selectively enhances the sound sources that a hearing-impaired listener is focusing their attention on. Here, we present and discuss a real-time brain-computer interface (BCI) system that combines a stimulus-response model based on canonical correlation analysis (CCA) for real-time EEG attention decoding, coupled with a multi-microphone hardware platform enabling low-latency real-time speech separation through spatial beamforming. We provide an overview of the system and its various components, discuss prospects and limitations of the technology, and illustrate its application with case studies of listeners steering acoustic feedback of competing speech streams via real-time attention decoding. A software implementation code of the system is publicly available for further research and explorations.

Neuroscience
Period effects on fertility for parity cohorts, Egypt: 1965-1980.

H. Rashad,O. El-issawy

1990-06-01

Abstract:"This paper presents a detailed investigation of period fertility trends in Egypt. It focusses on parity groups and changes in quantum and tempo of fertility of these groups. The analysis is further refined by controlling for age within each parity considered. The source of data...is the Egyptian Fertility Survey (EFS 80)....a retrospective fertility survey collected from a sample of 8,788 ever married Egyptian women in 1980."
Real-Time Tracking of Selective Auditory Attention From M/EEG: A Bayesian Filtering Approach

Sina Miran,Sahar Akram,Alireza Sheikhattar,Jonathan Z. Simon,Tao Zhang,Behtash Babadi

DOI: https://doi.org/10.3389/fnins.2018.00262

IF: 4.3

2018-05-01

Frontiers in Neuroscience

Abstract:Humans are able to identify and track a target speaker amid a cacophony of acoustic interference, an ability which is often referred to as the cocktail party phenomenon. Results from several decades of studying this phenomenon have culminated in recent years in various promising attempts to decode the attentional state of a listener in a competing-speaker environment from non-invasive neuroimaging recordings such as magnetoencephalography (MEG) and electroencephalography (EEG). To this end, most existing approaches compute correlation-based measures by either regressing the features of each speech stream to the M/EEG channels (the decoding approach) or vice versa (the encoding approach). To produce robust results, these procedures require multiple trials for training purposes. Also, their decoding accuracy drops significantly when operating at high temporal resolutions. Thus, they are not well-suited for emerging real-time applications such as smart hearing aid devices or brain-computer interface systems, where training data might be limited and high temporal resolutions are desired. In this paper, we close this gap by developing an algorithmic pipeline for real-time decoding of the attentional state. Our proposed framework consists of three main modules: (1) Real-time and robust estimation of encoding or decoding coefficients, achieved by sparse adaptive filtering, (2) Extracting reliable markers of the attentional state, and thereby generalizing the widely-used correlation-based measures thereof, and (3) Devising a near real-time state-space estimator that translates the noisy and variable attention markers to robust and statistically interpretable estimates of the attentional state with minimal delay. Our proposed algorithms integrate various techniques including forgetting factor-based adaptive filtering, ℓ<sub>1</sub>-regularization, forward-backward splitting algorithms, fixed-lag smoothing, and Expectation Maximization. We validate the performance of our proposed framework using comprehensive simulations as well as application to experimentally acquired M/EEG data. Our results reveal that the proposed real-time algorithms perform nearly as accurately as the existing state-of-the-art offline techniques, while providing a significant degree of adaptivity, statistical robustness, and computational savings.

neurosciences
EEG-based detection of the locus of auditory attention with convolutional neural networks

Servaas Vandecappelle,Lucas Deckers,Neetha Das,Amir Hossein Ansari,Alexander Bertrand,Tom Francart

DOI: https://doi.org/10.7554/eLife.56481

IF: 7.7

2021-04-30

eLife

Abstract:In a multi-speaker scenario, the human auditory system is able to attend to one particular speaker of interest and ignore the others. It has been demonstrated that it is possible to use electroencephalography (EEG) signals to infer to which speaker someone is attending by relating the neural activity to the speech signals. However, classifying auditory attention within a short time interval remains the main challenge. We present a convolutional neural network-based approach to extract the locus of auditory attention (left/right) without knowledge of the speech envelopes. Our results show that it is possible to decode the locus of attention within 1 to 2 s, with a median accuracy of around 81%. These results are promising for neuro-steered noise suppression in hearing aids, in particular in scenarios where per-speaker envelopes are unavailable.

biology
EEG-Based Auditory Attention Detection via Frequency and Channel Neural Attention

Siqi Cai,Enze Su,Longhan Xie,Haizhou Li

DOI: https://doi.org/10.1109/thms.2021.3125283

2022-04-01

Abstract:Humans have the ability to pay attention to one of the sound sources in a multispeaker acoustic environment. Auditory attention detection (AAD) seeks to detect the attended speaker from ones brain signals that will enable many innovative humanmachine systems. However, effective representation learning of electroencephalography (EEG) signals remains a challenge. In this article, we propose a neural attention mechanism that dynamically assigns differentiated weights to the subbands and the channels of EEG signals to derive discriminative representations for AAD. In the nutshell, we would like to build a computational attention mechanism, i.e., neural attention, to model the auditory attention in human brain. We incorporate the proposed neural attention into an AAD system, and validate the neural attention mechanism through comprehensive experiments on two publicly available datasets. The experimental results demonstrate that the proposed system significantly outperforms the state-of-the-art reference baselines.

computer science, cybernetics, artificial intelligence
Decoding of selective attention to continuous speech from the human auditory brainstem response

Octave Etard,Mikolaj Kegler,Chananel Braiman,Antonio Elia Forte,Tobias Reichenbach

DOI: https://doi.org/10.1101/259853

2018-02-05

Abstract:Abstract Humans are highly skilled at analysing complex acoustic scenes. The segregation of different acoustic streams and the formation of corresponding neural representations is mostly attributed to the auditory cortex. Decoding of selective attention from neuroimaging has therefore focussed on cortical responses to sound. However, the auditory brainstem response to speech is modulated by selective attention as well, as recently shown through measuring the brainstem’s response to running speech. Although the response of the auditory brainstem has a smaller magnitude than that of the auditory cortex, it occurs at much higher frequencies and therefore has a higher information rate. Here we develop statistical models for extracting the brainstem response from multi-channel scalp recordings and for analysing the attentional modulation according to the focus of attention. We demonstrate that the attentional modulation of the brainstem response to speech can be employed to decode the attentional focus of a listener from short measurements of ten seconds or less in duration. The decoding remains accurate when obtained from three EEG channels only. We further show how out-of-the-box decoding that employs subject-independent models, as well as decoding that is independent of the specific attended speaker is capable of achieving similar accuracy. These results open up new avenues for investigating the neural mechanisms for selective attention in the brainstem and for developing efficient auditory brain-computer interfaces.
Improved Decoding of Attentional Selection in Multi-Talker Environments with Self-Supervised Learned Speech Representation

Cong Han,Vishal Choudhari,Yinghao Aaron Li,Nima Mesgarani

DOI: https://doi.org/10.48550/arXiv.2302.05756

2023-02-12

Abstract:Auditory attention decoding (AAD) is a technique used to identify and amplify the talker that a listener is focused on in a noisy environment. This is done by comparing the listener's brainwaves to a representation of all the sound sources to find the closest match. The representation is typically the waveform or spectrogram of the sounds. The effectiveness of these representations for AAD is uncertain. In this study, we examined the use of self-supervised learned speech representation in improving the accuracy and speed of AAD. We recorded the brain activity of three subjects using invasive electrocorticography (ECoG) as they listened to two conversations and focused on one. We used WavLM to extract a latent representation of each talker and trained a spatiotemporal filter to map brain activity to intermediate representations of speech. During the evaluation, the reconstructed representation is compared to each speaker's representation to determine the target speaker. Our results indicate that speech representation from WavLM provides better decoding accuracy and speed than the speech envelope and spectrogram. Our findings demonstrate the advantages of self-supervised learned speech representation for auditory attention decoding and pave the way for developing brain-controlled hearable technologies.

Audio and Speech Processing,Sound,Signal Processing
EEG decoding of the target speaker in a cocktail party scenario: considerations regarding dynamic switching of talker location

Emily S Teoh,Edmund C Lalor

DOI: https://doi.org/10.1088/1741-2552/ab0cf1

IF: 5.043

2019-04-16

Journal of Neural Engineering

Abstract:OBJECTIVE: It has been shown that attentional selection in a simple dichotic listening paradigm can be decoded offline by reconstructing the stimulus envelope from single-trial neural response data. Here, we test the efficacy of this approach in an environment with non-stationary talkers. We then look beyond the envelope reconstructions themselves and consider whether incorporating the decoder values-which reflect the weightings applied to the multichannel EEG data at different time lags and scalp locations when reconstructing the stimulus envelope-can improve decoding performance.APPROACH: High-density EEG was recorded as subjects attended to one of two talkers. The two speech streams were filtered using HRTFs, and the talkers were alternated between the left and right locations at varying intervals to simulate a dynamic environment. We trained spatio-temporal decoders mapping from EEG data to the attended and unattended stimulus envelopes. We then decoded auditory attention by (1) using the attended decoder to reconstruct the envelope and (2) exploiting the fact that decoder weightings themselves contain signatures of attention, resulting in consistent patterns across subjects that can be classified.MAIN RESULTS: The previously established decoding approach was found to be effective even with non-stationary talkers. Signatures of attentional selection and attended direction were found in the spatio-temporal structure of the decoders and were consistent across subjects. The inclusion of decoder weights into the decoding algorithm resulted in significantly improved decoding accuracies (from 61.07% to 65.31% for 4 s windows). An attempt was made to include alpha power lateralization as another feature to improve decoding, although this was unsuccessful at the single-trial level.SIGNIFICANCE: This work suggests that the spatial-temporal decoder weights can be utilised to improve decoding. More generally, looking beyond envelope reconstruction and incorporating other signatures of attention is an avenue that should be explored to improve selective auditory attention decoding.

engineering, biomedical,neurosciences
EEG-based auditory attention decoding using speech-level-based segmented computational models

Lei Wang,Ed X Wu,Fei Chen

DOI: https://doi.org/10.1088/1741-2552/abfeba

IF: 5.043

2021-05-25

Journal of Neural Engineering

Abstract:Abstract Objective. Auditory attention in complex scenarios can be decoded by electroencephalography (EEG)-based cortical speech-envelope tracking. The relative root-mean-square (RMS) intensity is a valuable cue for the decomposition of speech into distinct characteristic segments. To improve auditory attention decoding (AAD) performance, this work proposed a novel segmented AAD approach to decode target speech envelopes from different RMS-level-based speech segments. Approach. Speech was decomposed into higher- and lower-RMS-level speech segments with a threshold of −10 dB relative RMS level. A support vector machine classifier was designed to identify higher- and lower-RMS-level speech segments, using clean target and mixed speech as reference signals based on corresponding EEG signals recorded when subjects listened to target auditory streams in competing two-speaker auditory scenes. Segmented computational models were developed with the classification results of higher- and lower-RMS-level speech segments. Speech envelopes were reconstructed based on segmented decoding models for either higher- or lower-RMS-level speech segments. AAD accuracies were calculated according to the correlations between actual and reconstructed speech envelopes. The performance of the proposed segmented AAD computational model was compared to those of traditional AAD methods with unified decoding functions. Main results. Higher- and lower-RMS-level speech segments in continuous sentences could be identified robustly with classification accuracies that approximated or exceeded 80% based on corresponding EEG signals at 6 dB, 3 dB, 0 dB, −3 dB and −6 dB signal-to-mask ratios (SMRs). Compared with unified AAD decoding methods, the proposed segmented AAD approach achieved more accurate results in the reconstruction of target speech envelopes and in the detection of attentional directions. Moreover, the proposed segmented decoding method had higher information transfer rates (ITRs) and shorter minimum expected switch times compared with the unified decoder. Significance. This study revealed that EEG signals may be used to classify higher- and lower-RMS-level-based speech segments across a wide range of SMR conditions (from 6 dB to −6 dB). A novel finding was that the specific information in different RMS-level-based speech segments facilitated EEG-based decoding of auditory attention. The significantly improved AAD accuracies and ITRs of the segmented decoding method suggests that this proposed computational model may be an effective method for the application of neuro-controlled brain–computer interfaces in complex auditory scenes.

engineering, biomedical,neurosciences
Auditory Attention Decoding in Four-Talker Environment with EEG

Yujie Yan,Xiran Xu,Haolin Zhu,Pei Tian,Zhongshu Ge,Xihong Wu,Jing Chen

DOI: https://doi.org/10.21437/interspeech.2024-739

2024-01-01

Abstract:Auditory Attention Decoding (AAD) is a technique that determines the focus of a listener's attention in complex auditory scenes according to cortical neural responses. Existing research largely examines two-talker scenarios, insufficient for real-world complexity. This study introduced a new AAD database for a four-talker scenario with speeches from four distinct talkers simultaneously presented and spatially separated, and listeners' EEG was recorded. Temporal response functions (TRFs) analysis showed that attended speech TRFs are stronger than each unattended speech. AAD methods based on stimulus-reconstruction (SR) and cortical spatial lateralization were employed and compared. Results indicated decoding accuracy of 77.5% in 60s (chance level of 25%) using SR. Using auditory spatial attention detection (ASAD) methods also indicated high accuracy (94.7% with DenseNet-3D in 1s), demonstrating ASAD methods' generalization performance.

Super-Arc-Connected and Super-Connected Total Digraphs

Decoding auditory attention (in real time) with eeg

Riemannian geometry-based decoding of the directional focus of auditory attention using EEG

What are we really decoding? Unveiling biases in EEG-based decoding of the spatial focus of auditory attention

Low Latency Auditory Attention Detection with Common Spatial Pattern Analysis of EEG Signals.

Using Ear-EEG to Decode Auditory Attention in Multiple-speaker Environment

Comparison of linear and nonlinear methods for decoding selective attention to speech from ear-EEG recordings

Electroencephalogram-based Multiclass Auditory Attention Decoding of Attended Speaker Direction

A DenseNet-based method for decoding auditory spatial attention with EEG

Deep learning-based auditory attention decoding in listeners with hearing impairment

Auditory attention decoding from electroencephalography based on long short-term memory networks

Real-time control of a hearing instrument with EEG-based attention decoding

Period effects on fertility for parity cohorts, Egypt: 1965-1980.

Real-Time Tracking of Selective Auditory Attention From M/EEG: A Bayesian Filtering Approach

EEG-based detection of the locus of auditory attention with convolutional neural networks

EEG-Based Auditory Attention Detection via Frequency and Channel Neural Attention

Decoding of selective attention to continuous speech from the human auditory brainstem response

Improved Decoding of Attentional Selection in Multi-Talker Environments with Self-Supervised Learned Speech Representation

EEG decoding of the target speaker in a cocktail party scenario: considerations regarding dynamic switching of talker location

EEG-based auditory attention decoding using speech-level-based segmented computational models

Auditory Attention Decoding in Four-Talker Environment with EEG