Optimising EEG decoding with refined sampling and multimodal feature integration

Arash Akbarinia

2024-09-30

Abstract:Electroencephalography (EEG) is a neuroimaging technique that records brain neural activity with high temporal resolution. Unlike other methods, EEG does not require prohibitively expensive equipment and can be easily set up using commercially available portable EEG caps, making it an ideal candidate for brain-computer interfaces. However, EEG signals are characterised by poor spatial resolution and high noise levels, complicating their decoding. In this study, we employ a contrastive learning framework to align encoded EEG features with pretrained CLIP features, achieving a 7% improvement over the state-of-the-art in EEG decoding of object categories. This enhancement is equally attributed to (1) a novel online sampling method that boosts the signal-to-noise ratio and (2) multimodal representations leveraging visual and language features to enhance the alignment space. Our analysis reveals a systematic interaction between the architecture and dataset of pretrained features and their alignment efficacy for EEG signal decoding. This interaction correlates with the generalisation power of the pretrained features on ImageNet-O/A datasets ($r=.5$). These findings extend beyond EEG signal alignment, offering potential for broader applications in neuroimaging decoding and generic feature alignments.

Human-Computer Interaction

What problem does this paper attempt to address?

The paper aims to address several key issues in EEG (Electroencephalogram) signal decoding. Specifically: 1. **Improving Signal-to-Noise Ratio**: EEG signals have a high noise level and poor spatial resolution, making decoding complex. The paper proposes a new online sampling method (InterDimensional EEG Sampling, abbreviated as IDES) to enhance the signal-to-noise ratio by expanding the training space. 2. **Multimodal Feature Fusion**: To further enhance decoding performance, the paper introduces multimodal features (visual and language features) for alignment, utilizing these features to improve the decoding space. 3. **Reducing Overfitting**: Existing advanced methods still suffer from overfitting in practical applications, leading to insufficient generalization ability. Although the proposed framework cannot completely eliminate overfitting, it significantly reduces this issue, thereby improving test accuracy. Through the above methods, the paper achieves approximately a 7% performance improvement over existing technologies, and this improvement is consistent across different pre-training architectures. The research results indicate that the new method is not only applicable to EEG signal decoding but may also extend to other neuroimaging decoding tasks.

Optimising EEG decoding with refined sampling and multimodal feature integration

Extracting Semantic-Dynamic Features for Long-Term Stable Brain Computer Interface.

Multimodal Speech Recognition Using EEG and Audio Signals: A Novel Approach for Enhancing ASR Systems

Decoding Natural Images from EEG for Object Recognition

A Hybrid Autoencoder Framework of Dimensionality Reduction for Brain-Computer Interface Decoding

Transformer-based Spatial-Temporal Feature Learning for EEG Decoding

Improving EEG Decoding via Clustering-based Multi-task Feature Learning

EEG classification for visual brain decoding with spatio-temporal and transformer based paradigms

Can Brain Signals Reveal Inner Alignment with Human Languages?

Latent Alignment with Deep Set EEG Decoders

A Combined Channel Approach for Decoding Intracranial EEG Signals: Enhancing Accuracy through Spatial Information Integration

Combining Euclidean Alignment and Data Augmentation for BCI decoding

Enhancing EEG-to-Text Decoding through Transferable Representations from Pre-trained Contrastive EEG-Text Masked Autoencoder

Multi-domain feature joint optimization based on multi-view learning for improving the EEG decoding

Improving EEG Decoding via Clustering-Based Multitask Feature Learning

Combining Statistical Analysis and Machine Learning for EEG Scalp Topograms Classification

Feature optimization based on improved novel global harmony search algorithm for motor imagery electroencephalogram classification

Optimization of transfer learning based on source sample selection in Euclidean space for P300-based brain-computer interfaces

Decoding Brain Signals from Rapid-Event EEG for Visual Analysis Using Deep Learning

Visual Decoding and Reconstruction via EEG Embeddings with Guided Diffusion

Learning joint space-time-frequency features for EEG decoding on small labeled data