[The treatment of fractures of the radial condyle of the humerus during the growth phase].

L. V. von Laer,P. Pagels,L. Schroeder

1983-12-01

Abstract:

What problem does this paper attempt to address?

Deep learning-based auditory attention decoding in listeners with hearing impairment

M Asjid Tanveer,Martin A Skoglund,Bo Bernhardsson,Emina Alickovic

DOI: https://doi.org/10.1088/1741-2552/ad49d7

IF: 5.043

2024-05-12

Journal of Neural Engineering

Abstract:Objective: This study develops a deep learning method for fast auditory attention decoding (AAD) using electroencephalography (EEG) from listeners with hearing impairment. It addresses three classification tasks: differentiating noise from speech-in-noise, classifying the direction of attended speech (left vs. right) and identifying the activation status of hearing aid noise reduction (NR) algorithms (OFF vs. ON). These tasks contribute to our understanding of how hearing technology influences auditory processing in the hearing-impaired population. Method: Deep convolutional neural network (DCNN) models were designed for each task. Two training strategies were employed to clarify the impact of data splitting on AAD tasks: inter-trial, where the testing set used classification windows from trials that the training set hadn't seen, and intra-trial, where the testing set used unseen classification windows from trials where other segments were seen during training. The models were evaluated on EEG data from 31 participants with hearing impairment, listening to competing talkers amidst background noise. Results: Using 1-second classification windows, DCNN models achieve accuracy (ACC) of 69.8\%, 73.3\% and 82.9\% and area-under-curve (AUC) of 77.2\%, 80.6\% and 92.1\% for the three tasks respectively on inter-trial strategy. In the intra-trial strategy, they achieved ACC of 87.9\%, 80.1\% and 97.5\%, along with AUC of 94.6\%, 89.1\%, and 99.8\%. Our DCNN models show good performance on short 1-second EEG samples, making them suitable for real-world applications. Conclusion: Our DCNN models successfully addressed three tasks with short 1-second EEG windows from participants with hearing impairment, showcasing their potential. While the inter-trial strategy demonstrated promise for assessing AAD, the intra-trial approach yielded inflated results, underscoring the important role of proper data splitting in EEG-based AAD tasks. Significance: Our findings showcase the promising potential of EEG-based tools for assessing auditory attention in clinical contexts and advancing hearing technology, while also promoting further exploration of alternative deep learning architectures and their potential constraints.

engineering, biomedical,neurosciences
EEG-informed attended speaker extraction from recorded speech mixtures with application in neuro-steered hearing prostheses

Simon Van Eyndhoven,Tom Francart,Alexander Bertrand

DOI: https://doi.org/10.1109/TBME.2016.2587382

2019-02-05

Abstract:OBJECTIVE: We aim to extract and denoise the attended speaker in a noisy, two-speaker acoustic scenario, relying on microphone array recordings from a binaural hearing aid, which are complemented with electroencephalography (EEG) recordings to infer the speaker of interest. METHODS: In this study, we propose a modular processing flow that first extracts the two speech envelopes from the microphone recordings, then selects the attended speech envelope based on the EEG, and finally uses this envelope to inform a multi-channel speech separation and denoising algorithm. RESULTS: Strong suppression of interfering (unattended) speech and background noise is achieved, while the attended speech is preserved. Furthermore, EEG-based auditory attention detection (AAD) is shown to be robust to the use of noisy speech signals. CONCLUSIONS: Our results show that AAD-based speaker extraction from microphone array recordings is feasible and robust, even in noisy acoustic environments, and without access to the clean speech signals to perform EEG-based AAD. SIGNIFICANCE: Current research on AAD always assumes the availability of the clean speech signals, which limits the applicability in real settings. We have extended this research to detect the attended speaker even when only microphone recordings with noisy speech mixtures are available. This is an enabling ingredient for new brain-computer interfaces and effective filtering schemes in neuro-steered hearing prostheses. Here, we provide a first proof of concept for EEG-informed attended speaker extraction and denoising.

Sound,Systems and Control,Machine Learning
Comparison of linear and nonlinear methods for decoding selective attention to speech from ear-EEG recordings

Mike Thornton,Danilo Mandic,Tobias Reichenbach

2024-11-15

Abstract:Many people with hearing loss struggle to comprehend speech in crowded auditory scenes, even when they are using hearing aids. It has recently been demonstrated that the focus of a listener's selective attention to speech can be decoded from their electroencephalography (EEG) recordings, raising the prospect of smart EEG-steered hearing aids which restore speech comprehension in adverse acoustic environments (such as the cocktail party). To this end, we here assess the feasibility of using a novel, ultra-wearable ear-EEG device to classify the selective attention of normal-hearing listeners who participated in a two-talker competing-speakers experiment. Eighteen participants took part in a diotic listening task, whereby they were asked to attend to one narrator whilst ignoring the other. Encoding models were estimated from the recorded signals, and these confirmed that the device has the ability to capture auditory responses that are consistent with those reported in high-density EEG studies. Several state-of-the-art auditory attention decoding algorithms were next compared, including stimulus-reconstruction algorithms based on linear regression as well as non-linear deep neural networks, and canonical correlation analysis (CCA). Meaningful markers of selective auditory attention could be extracted from the ear-EEG signals of all 18 participants, even when those markers were derived from relatively short EEG segments of just five seconds in duration. Algorithms which related the EEG signals to the rising edges of the speech temporal envelope (onset envelope) were more successful than those which made use of the temporal envelope itself. The CCA algorithm achieved the highest mean attention decoding accuracy, although differences between the performances of the three algorithms were both small and not statistically significant when EEG segments of short durations were employed.

Audio and Speech Processing
Robust decoding of the speech envelope from EEG recordings through deep neural networks

Mike Thornton,Danilo Mandic,Tobias Reichenbach

DOI: https://doi.org/10.1088/1741-2552/ac7976

IF: 5.043

2022-06-17

Journal of Neural Engineering

Abstract:Objective. Smart hearing aids which can decode the focus of a user's attention could considerably improve comprehension levels in noisy environments. Methods for decoding auditory attention from electroencephalography (EEG) have attracted considerable interest for this reason. Recent studies suggest that the integration of deep neural networks (DNNs) into existing auditory attention decoding algorithms is highly beneficial, although it remains unclear whether these enhanced algorithms can perform robustly in different real-world scenarios. To this end, we sought to characterise the performance of DNNs at reconstructing the envelope of an attended speech stream from EEG recordings in different listening conditions. In addition, given the relatively sparse availability of EEG data, we investigate possibility of applying subject-independent algorithms to EEG recorded from unseen individuals. Approach. Both linear models and nonlinear DNNs were employed to decode the envelope of clean speech from EEG recordings, with and without subject-specific information. The mean behaviour, as well as the variability of the reconstruction, was characterised for each model. We then trained subject-specific linear models and DNNs to reconstruct the envelope of speech in clean and noisy conditions, and investigated how well they performed in different listening scenarios. We also established that these models can be used to decode auditory attention in competing-speaker scenarios. Main results. The DNNs offered a considerable advantage over their linear counterpart at reconstructing the envelope of clean speech. This advantage persisted even when subject-specific information was unavailable at the time of training. The same DNN architectures generalised to a distinct dataset, which contained EEG recorded under a variety of listening conditions. In competing-speakers and speech-in-noise conditions, the DNNs significantly outperformed the linear models. Finally, the DNNs offered a considerable improvement over the linear approach at decoding auditory attention in competing-speakers scenarios. Significance. We present the first detailed study into the extent to which DNNs can be employed for reconstructing the envelope of an attended speech stream. We conclusively demonstrate that DNNs have the ability to improve the reconstruction of the attended speech envelope. The variance of the reconstruction error is shown to be similar for both DNNs and the linear model. Overall, DNNs are demonstrated to show promise for real-world auditory attention decoding, since they perform well in multiple listening conditions and generalise to data recorded from unseen participants.

engineering, biomedical,neurosciences
Period effects on fertility for parity cohorts, Egypt: 1965-1980.

H. Rashad,O. El-issawy

1990-06-01

Abstract:"This paper presents a detailed investigation of period fertility trends in Egypt. It focusses on parity groups and changes in quantum and tempo of fertility of these groups. The analysis is further refined by controlling for age within each parity considered. The source of data...is the Egyptian Fertility Survey (EFS 80)....a retrospective fertility survey collected from a sample of 8,788 ever married Egyptian women in 1980."
Hunting the elusive oncogene: a stroke of good luck

R. Weinberg

DOI: https://doi.org/10.1038/ncb2302

IF: 21.3

2011-08-01

Nature Cell Biology

Abstract:
Speaker separation in realistic noise environments with applications to a cognitively-controlled hearing aid

Bengt J Borgström,Michael S Brandstein,Gregory A Ciccarelli,Thomas F Quatieri,Christopher J Smalt

DOI: https://doi.org/10.1016/j.neunet.2021.02.020

Abstract:Future wearable technology may provide for enhanced communication in noisy environments and for the ability to pick out a single talker of interest in a crowded room simply by the listener shifting their attentional focus. Such a system relies on two components, speaker separation and decoding the listener's attention to acoustic streams in the environment. To address the former, we present a system for joint speaker separation and noise suppression, referred to as the Binaural Enhancement via Attention Masking Network (BEAMNET). The BEAMNET system is an end-to-end neural network architecture based on self-attention. Binaural input waveforms are mapped to a joint embedding space via a learned encoder, and separate multiplicative masking mechanisms are included for noise suppression and speaker separation. Pairs of output binaural waveforms are then synthesized using learned decoders, each capturing a separated speaker while maintaining spatial cues. A key contribution of BEAMNET is that the architecture contains a separation path, an enhancement path, and an autoencoder path. This paper proposes a novel loss function which simultaneously trains these paths, so that disabling the masking mechanisms during inference causes BEAMNET to reconstruct the input speech signals. This allows dynamic control of the level of suppression applied by BEAMNET via a minimum gain level, which is not possible in other state-of-the-art approaches to end-to-end speaker separation. This paper also proposes a perceptually-motivated waveform distance measure. Using objective speech quality metrics, the proposed system is demonstrated to perform well at separating two equal-energy talkers, even in high levels of background noise. Subjective testing shows an improvement in speech intelligibility across a range of noise levels, for signals with artificially added head-related transfer functions and background noise. Finally, when used as part of an auditory attention decoder (AAD) system using existing electroencephalogram (EEG) data, BEAMNET is found to maintain the decoding accuracy achieved with ideal speaker separation, even in severe acoustic conditions. These results suggest that this enhancement system is highly effective at decoding auditory attention in realistic noise environments, and could possibly lead to improved speech perception in a cognitively controlled hearing aid.
AADNet: An End-to-End Deep Learning Model for Auditory Attention Decoding

Nhan Duc Thanh Nguyen,Huy Phan,Simon Geirnaert,Kaare Mikkelsen,Preben Kidmose

2024-10-17

Abstract:Auditory attention decoding (AAD) is the process of identifying the attended speech in a multi-talker environment using brain signals, typically recorded through electroencephalography (EEG). Over the past decade, AAD has undergone continuous development, driven by its promising application in neuro-steered hearing devices. Most AAD algorithms are relying on the increase in neural entrainment to the envelope of attended speech, as compared to unattended speech, typically using a two-step approach. First, the algorithm predicts representations of the attended speech signal envelopes; second, it identifies the attended speech by finding the highest correlation between the predictions and the representations of the actual speech signals. In this study, we proposed a novel end-to-end neural network architecture, named AADNet, which combines these two stages into a direct approach to address the AAD problem. We compare the proposed network against the traditional approaches, including linear stimulus reconstruction, canonical correlation analysis, and an alternative non-linear stimulus reconstruction using two different datasets. AADNet shows a significant performance improvement for both subject-specific and subject-independent models. Notably, the average subject-independent classification accuracies from 56.1 % to 82.7 % with analysis window lengths ranging from 1 to 40 seconds, respectively, show a significantly improved ability to generalize to data from unseen subjects. These results highlight the potential of deep learning models for advancing AAD, with promising implications for future hearing aids, assistive devices, and clinical assessments.

Sound,Audio and Speech Processing,Signal Processing
Cognitive-Driven Binaural Beamforming Using EEG-Based Auditory Attention Decoding

Ali Aroudi,Simon Doclo

DOI: https://doi.org/10.1109/taslp.2020.2969779

2020-01-01

Abstract:Identifying the target speaker in hearing aid applications is an essential ingredient to improve speech intelligibility. Recently, a least-squares-based auditory attention decoding (AAD) method has been proposed to identify the target speaker from single-trial EEG recordings in an acoustic scenario with two competing speakers. Aiming at enhancing the target speaker and suppressing the interfering speaker and ambient noise, in this article, we propose a cognitive-driven speech enhancement system, consisting of a binaural beamformer which is steered based on AAD and estimated relative transfer function (RTF) vectors, which require estimates of the direction-of-arrivals (DOAs) of both speakers. For binaural beamforming and to generate reference signals for AAD, we consider either minimum-variance-distortionless-response (MVDR) beamformers or linearly-constrained-minimum-variance (LCMV) beamformers. Contrary to the binaural MVDR beamformer, the binaural LCMV beamformer allows to preserve the spatial impression of the acoustic scene and to control the suppression of the interfering speaker, which is important when intending to switch attention between speakers. The speech enhancement performance of the proposed system is evaluated in terms of the binaural signal-to-interference-plus-noise ratio ($text {SINR}$) improvement in anechoic and reverberant conditions. Furthermore, we investigate the impact of RTF and DOA estimation errors and AAD errors on the speech enhancement performance. The experimental results show that the proposed system using LCMV beamformers yields a larger decoding performance and binaural $text {SINR}$ improvement compared to using MVDR beamformers.
Improving auditory attention decoding performance of linear and non-linear methods using state-space model

Ali Aroudi,Tobias de Taillez,Simon Doclo

DOI: https://doi.org/10.48550/arXiv.2004.00910

2020-04-02

Abstract:Identifying the target speaker in hearing aid applications is crucial to improve speech understanding. Recent advances in electroencephalography (EEG) have shown that it is possible to identify the target speaker from single-trial EEG recordings using auditory attention decoding (AAD) methods. AAD methods reconstruct the attended speech envelope from EEG recordings, based on a linear least-squares cost function or non-linear neural networks, and then directly compare the reconstructed envelope with the speech envelopes of speakers to identify the attended speaker using Pearson correlation coefficients. Since these correlation coefficients are highly fluctuating, for a reliable decoding a large correlation window is used, which causes a large processing delay. In this paper, we investigate a state-space model using correlation coefficients obtained with a small correlation window to improve the decoding performance of the linear and the non-linear AAD methods. The experimental results show that the state-space model significantly improves the decoding performance.

Audio and Speech Processing,Machine Learning,Sound,Signal Processing
Real-time control of a hearing instrument with EEG-based attention decoding

Jens Hjortkjaer,Daniel D.E. Wong,Alessandro Catania,Jonatan Marcher-Rorsted,Enea Ceolini,Soren Asp Fuglsang,Ilya Kiselev,Giovanni Di Liberto,Shih-Chii Liu,Torsten Dau,Malcolm Slaney,Alain de Cheveigne

DOI: https://doi.org/10.1101/2024.03.01.582668

2024-06-12

Abstract:Enhancing speech perception in everyday noisy acoustic environments remains an outstanding challenge for hearing aids. Speech separation technology is improving rapidly, but hearing devices cannot fully exploit this advance without knowing which sound sources the user wants to hear. Even with high-quality source separation, the hearing aid must know which speech streams to enhance and which to suppress. Advances in EEG-based decoding of auditory attention raise the potential of neurosteering, in which a hearing instrument selectively enhances the sound sources that a hearing-impaired listener is focusing their attention on. Here, we present and discuss a real-time brain-computer interface (BCI) system that combines a stimulus-response model based on canonical correlation analysis (CCA) for real-time EEG attention decoding, coupled with a multi-microphone hardware platform enabling low-latency real-time speech separation through spatial beamforming. We provide an overview of the system and its various components, discuss prospects and limitations of the technology, and illustrate its application with case studies of listeners steering acoustic feedback of competing speech streams via real-time attention decoding. A software implementation code of the system is publicly available for further research and explorations.

Neuroscience
Decoding the Unintelligible: Neural Speech Tracking in Low Signal-to-Noise Ratios

Xiaomin He,Vinay S Raghavan,Nima Mesgarani

DOI: https://doi.org/10.1101/2024.10.10.616521

2024-10-12

Abstract:Understanding speech in noisy environments is challenging for both human listeners and technology, with significant implications for hearing aid design and communication systems. Auditory attention decoding (AAD) aims to decode the attended talker from neural signals to enhance their speech and improve intelligibility. However, whether this decoding remains reliable when speech intelligibility is severely degraded in real-world listening conditions remains unclear. In this study, we investigated selective neural tracking of the attended speaker under adverse listening conditions. Using EEG recordings in a multi-talker speech perception task with varying SNR, participants' speech perception was assessed through a repeated-word detection task, while neural responses were analyzed to decode the attended talker. Despite substantial degradation in intelligibility, we found that neural tracking of attended speech persists, suggesting that the brain retains sufficient information for decoding. These findings demonstrate that even in highly challenging conditions, AAD remains feasible, offering a potential avenue for enhancing speech intelligibility in brain-informed audio technologies, such as hearing aids, that leverage AAD to improve speech perception in real-world noisy environments.

Neuroscience
Auditory attention decoding from electroencephalography based on long short-term memory networks

Yun Lu,Mingjiang Wang,Longxin Yao,Hongcai Shen,Wanqing Wu,Qiquan Zhang,Lu Zhang,Moran Chen,Hao Liu,Rongchao Peng,Ming Liu,Shixiong Chen

DOI: https://doi.org/10.1016/j.bspc.2021.102966

IF: 5.1

2021-09-01

Biomedical Signal Processing and Control

Abstract:<p>Available researches have confirmed that the target speech stream of human listener's selective attention in a multi-speaker scenario can be decoded from ongoing electroencephalography (EEG) signals. Due to its great potential application for the neuro-steered hearing devices, auditory attention decoding has recently received increasing attention. In practice, development of high-accuracy decoding performance and use of EEG signals with as short duration as possible are urgently needed for auditory attention decoding in real-time scenarios. However, these are still great challenges for the existing decoding methods to achieve such performances. In this paper, we propose a nonlinear modeling approach between EEG signals and speech envelopes by exploiting long short-term memory networks (LSTM), a special structure of recurrent neural networks (RNN), to decipher selective auditory attention from single-trial EEG signals, which is capable of learning long-term dependencies between EEG responses and continuous auditory stimuli. Unlike the existing linear modeling approaches, the LSTM RNN-based auditory attention decoding method is an end-to-end nonlinear decoding framework that subsumes the explicit similarity determination as part of feature learning in deep neural networks. The proposed decoding method is evaluated using the real EEG data collected from 21 subjects. Experimental results show that our proposed decoding method can achieve the state-of-the-art performance to identify the target speech in a competing two-talker scenario. When used the EEG signals with 1 s duration, the average accuracy for the 21 subjects' attention decoding is reached to 96.12%. The results show great potentials for the real-time auditory attention detection in neuro-steered hearing aids.</p>

engineering, biomedical
Investigating Self-Supervised Deep Representations for EEG-based Auditory Attention Decoding

Karan Thakkar,Jiarui Hai,Mounya Elhilali

2023-11-08

Abstract:Auditory Attention Decoding (AAD) algorithms play a crucial role in isolating desired sound sources within challenging acoustic environments directly from brain activity. Although recent research has shown promise in AAD using shallow representations such as auditory envelope and spectrogram, there has been limited exploration of deep Self-Supervised (SS) representations on a larger scale. In this study, we undertake a comprehensive investigation into the performance of linear decoders across 12 deep and 2 shallow representations, applied to EEG data from multiple studies spanning 57 subjects and multiple languages. Our experimental results consistently reveal the superiority of deep features for AAD at decoding background speakers, regardless of the datasets and analysis windows. This result indicates possible nonlinear encoding of unattended signals in the brain that are revealed using deep nonlinear features. Additionally, we analyze the impact of different layers of SS representations and window sizes on AAD performance. These findings underscore the potential for enhancing EEG-based AAD systems through the integration of deep feature representations.

Sound,Audio and Speech Processing
An Interpretable Performance Metric for Auditory Attention Decoding Algorithms in a Context of Neuro-Steered Gain Control

Simon Geirnaert,Tom Francart,Alexander Bertrand

DOI: https://doi.org/10.1101/745695

2019-08-24

Abstract:Abstract In a multi-speaker scenario, a hearing aid lacks information on which speaker the user intends to attend, and therefore it often mistakenly treats the latter as noise while enhancing an interfering speaker. Recently, it has been shown that it is possible to decode the attended speaker from the brain activity, e.g., recorded by electroencephalography sensors. While numerous of these auditory attention decoding (AAD) algorithms appeared in the literature, their performance is generally evaluated in a non-uniform manner. Furthermore, AAD algorithms typically introduce a trade-off between the AAD accuracy and the time needed to make an AAD decision, which hampers an objective benchmarking as it remains unclear which point in each algorithm’s trade-off space is the optimal one in a context of neuro-steered gain control. To this end, we present an interpretable performance metric to evaluate AAD algorithms, based on an adaptive gain control system, steered by AAD decisions. Such a system can be modeled as a Markov chain, from which the minimal expected switch duration (MESD) can be calculated and interpreted as the expected time required to switch the operation of the hearing aid after an attention switch of the user, thereby resolving the trade-off between AAD accuracy and decision time. Furthermore, we show that the MESD calculation provides an automatic and theoretically founded procedure to optimize the number of gain levels and decision time in an AAD-based adaptive gain control system.
Towards auditory attention decoding with noise-tagging: A pilot study

H. A. Scheppink,S. Ahmadi,P. Desain,M. Tangermann,J. Thielen

2024-05-17

Abstract:Auditory attention decoding (AAD) aims to extract from brain activity the attended speaker amidst candidate speakers, offering promising applications for neuro-steered hearing devices and brain-computer interfacing. This pilot study makes a first step towards AAD using the noise-tagging stimulus protocol, which evokes reliable code-modulated evoked potentials, but is minimally explored in the auditory modality. Participants were sequentially presented with two Dutch speech stimuli that were amplitude-modulated with a unique binary pseudo-random noise-code, effectively tagging these with additional decodable information. We compared the decoding of unmodulated audio against audio modulated with various modulation depths, and a conventional AAD method against a standard method to decode noise-codes. Our pilot study revealed higher performances for the conventional method with 70 to 100 percent modulation depths compared to unmodulated audio. The noise-code decoder did not further improve these results. These fundamental insights highlight the potential of integrating noise-codes in speech to enhance auditory speaker detection when multiple speakers are presented simultaneously.

Neurons and Cognition,Artificial Intelligence,Machine Learning,Sound,Audio and Speech Processing
Super-Arc-Connected and Super-Connected Total Digraphs

Juan Liu,J. Meng

Abstract:
Predicting EEG Responses to Attended Speech via Deep Neural Networks for Speech

Emina Alickovic,Tobias Dorszewski,Thomas U. Christiansen,Kasper Eskelund,Leonardo Gizzi,Martin A. Skoglund,Dorothea Wendt

DOI: https://doi.org/10.48550/arXiv.2302.13553

2023-02-27

Abstract:Attending to the speech stream of interest in multi-talker environments can be a challenging task, particularly for listeners with hearing impairment. Research suggests that neural responses assessed with electroencephalography (EEG) are modulated by listener`s auditory attention, revealing selective neural tracking (NT) of the attended speech. NT methods mostly rely on hand-engineered acoustic and linguistic speech features to predict the neural response. Only recently, deep neural network (DNN) models without specific linguistic information have been used to extract speech features for NT, demonstrating that speech features in hierarchical DNN layers can predict neural responses throughout the auditory pathway. In this study, we go one step further to investigate the suitability of similar DNN models for speech to predict neural responses to competing speech observed in EEG. We recorded EEG data using a 64-channel acquisition system from 17 listeners with normal hearing instructed to attend to one of two competing talkers. Our data revealed that EEG responses are significantly better predicted by DNN-extracted speech features than by hand-engineered acoustic features. Furthermore, analysis of hierarchical DNN layers showed that early layers yielded the highest predictions. Moreover, we found a significant increase in auditory attention classification accuracies with the use of DNN-extracted speech features over the use of hand-engineered acoustic features. These findings open a new avenue for development of new NT measures to evaluate and further advance hearing technology.

Human-Computer Interaction,Sound,Audio and Speech Processing
EEG-based auditory attention decoding using speech-level-based segmented computational models

Lei Wang,Ed X Wu,Fei Chen

DOI: https://doi.org/10.1088/1741-2552/abfeba

IF: 5.043

2021-05-25

Journal of Neural Engineering

Abstract:Abstract Objective. Auditory attention in complex scenarios can be decoded by electroencephalography (EEG)-based cortical speech-envelope tracking. The relative root-mean-square (RMS) intensity is a valuable cue for the decomposition of speech into distinct characteristic segments. To improve auditory attention decoding (AAD) performance, this work proposed a novel segmented AAD approach to decode target speech envelopes from different RMS-level-based speech segments. Approach. Speech was decomposed into higher- and lower-RMS-level speech segments with a threshold of −10 dB relative RMS level. A support vector machine classifier was designed to identify higher- and lower-RMS-level speech segments, using clean target and mixed speech as reference signals based on corresponding EEG signals recorded when subjects listened to target auditory streams in competing two-speaker auditory scenes. Segmented computational models were developed with the classification results of higher- and lower-RMS-level speech segments. Speech envelopes were reconstructed based on segmented decoding models for either higher- or lower-RMS-level speech segments. AAD accuracies were calculated according to the correlations between actual and reconstructed speech envelopes. The performance of the proposed segmented AAD computational model was compared to those of traditional AAD methods with unified decoding functions. Main results. Higher- and lower-RMS-level speech segments in continuous sentences could be identified robustly with classification accuracies that approximated or exceeded 80% based on corresponding EEG signals at 6 dB, 3 dB, 0 dB, −3 dB and −6 dB signal-to-mask ratios (SMRs). Compared with unified AAD decoding methods, the proposed segmented AAD approach achieved more accurate results in the reconstruction of target speech envelopes and in the detection of attentional directions. Moreover, the proposed segmented decoding method had higher information transfer rates (ITRs) and shorter minimum expected switch times compared with the unified decoder. Significance. This study revealed that EEG signals may be used to classify higher- and lower-RMS-level-based speech segments across a wide range of SMR conditions (from 6 dB to −6 dB). A novel finding was that the specific information in different RMS-level-based speech segments facilitated EEG-based decoding of auditory attention. The significantly improved AAD accuracies and ITRs of the segmented decoding method suggests that this proposed computational model may be an effective method for the application of neuro-controlled brain–computer interfaces in complex auditory scenes.

engineering, biomedical,neurosciences
Single-word Auditory Attention Decoding Using Deep Learning Model

Nhan Duc Thanh Nguyen,Huy Phan,Kaare Mikkelsen,Preben Kidmose

2024-10-16

Abstract:Identifying auditory attention by comparing auditory stimuli and corresponding brain responses, is known as auditory attention decoding (AAD). The majority of AAD algorithms utilize the so-called envelope entrainment mechanism, whereby auditory attention is identified by how the envelope of the auditory stream drives variation in the electroencephalography (EEG) signal. However, neural processing can also be decoded based on endogenous cognitive responses, in this case, neural responses evoked by attention to specific words in a speech stream. This approach is largely unexplored in the field of AAD but leads to a single-word auditory attention decoding problem in which an epoch of an EEG signal timed to a specific word is labeled as attended or unattended. This paper presents a deep learning approach, based on EEGNet, to address this challenge. We conducted a subject-independent evaluation on an event-based AAD dataset with three different paradigms: word category oddball, word category with competing speakers, and competing speech streams with targets. The results demonstrate that the adapted model is capable of exploiting cognitive-related spatiotemporal EEG features and achieving at least 58% accuracy on the most realistic competing paradigm for the unseen subjects. To our knowledge, this is the first study dealing with this problem.

Signal Processing,Artificial Intelligence,Human-Computer Interaction,Sound,Audio and Speech Processing,Neurons and Cognition

[The treatment of fractures of the radial condyle of the humerus during the growth phase].

Deep learning-based auditory attention decoding in listeners with hearing impairment

EEG-informed attended speaker extraction from recorded speech mixtures with application in neuro-steered hearing prostheses

Comparison of linear and nonlinear methods for decoding selective attention to speech from ear-EEG recordings

Robust decoding of the speech envelope from EEG recordings through deep neural networks

Period effects on fertility for parity cohorts, Egypt: 1965-1980.

Hunting the elusive oncogene: a stroke of good luck

Speaker separation in realistic noise environments with applications to a cognitively-controlled hearing aid

AADNet: An End-to-End Deep Learning Model for Auditory Attention Decoding

Cognitive-Driven Binaural Beamforming Using EEG-Based Auditory Attention Decoding

Improving auditory attention decoding performance of linear and non-linear methods using state-space model

Real-time control of a hearing instrument with EEG-based attention decoding

Decoding the Unintelligible: Neural Speech Tracking in Low Signal-to-Noise Ratios

Auditory attention decoding from electroencephalography based on long short-term memory networks

Investigating Self-Supervised Deep Representations for EEG-based Auditory Attention Decoding

An Interpretable Performance Metric for Auditory Attention Decoding Algorithms in a Context of Neuro-Steered Gain Control

Towards auditory attention decoding with noise-tagging: A pilot study

Super-Arc-Connected and Super-Connected Total Digraphs

Predicting EEG Responses to Attended Speech via Deep Neural Networks for Speech

EEG-based auditory attention decoding using speech-level-based segmented computational models

Single-word Auditory Attention Decoding Using Deep Learning Model