Abstract:Objective. Functional near-infrared spectroscopy (fNIRS) can measure neural activity through blood oxygenation changes in the brain in a wearable form factor, enabling unique applications for research in and outside the lab and in practical occupational settings. fNIRS has proven capable of measuring cognitive states such as mental workload, often using machine learning (ML) based brain-computer interfaces (BCIs). To date, this research has largely relied on probes with channel counts from under ten to several hundred, although recently a new class of wearable NIRS devices featuring thousands of channels has emerged. This poses unique challenges for ML classification, as fNIRS is typically limited by few training trials, which results in severely under-determined estimation problems. So far, it is not well understood how such high-resolution data is best leveraged in practical BCIs and whether state-of-the-art or better performance can be achieved. Approach. To address these questions, we propose an ML strategy to classify working-memory load that relies on spatio-temporal regularization and transfer learning from other subjects in a combination that, to our knowledge, has not been used in previous fNIRS BCIs. The approach can be interpreted as an end-to-end generalized linear model and allows for a high degree of interpretability using channel-level or cortical imaging approaches. Main Results. We show that using the proposed methodology, it is possible to achieve state-of-the-art decoding performance with high-resolution fNIRS data. We also replicated several state-of-the-art approaches on our dataset of 43 participants wearing a 3198 dual-channel NIRS device while performing the n-Back task and show that these existing methodologies struggle in the high-channel regime and are largely outperformed by the proposed pipeline. Significance. Our approach helps establish high-channel NIRS devices as a viable platform for state-of-the-art BCI and opens new applications using this class of headset while also enabling high-resolution model imaging and interpretation.

Working memory load recognition with deep learning time series classification

Taking a Deeper Look at the Brain: Predicting Visual Perceptual and Working Memory Load From High-Density fNIRS Data

Decoding Working Memory Load from EEG with LSTM Networks

Decoding Working Memory Representation Based on Single-Trial EEG by a Deep Learning Framework

Decoding working-memory load during n-Back task performance from high channel fNIRS data

Cognitive Analysis of Working Memory Load from Eeg, by a Deep Recurrent Neural Network

Task State Decoding and Mapping of Individual Four-Dimensional Fmri Time Series Using Deep Neural Network

Identification of Temporal Variations in Mental Workload Using Locally-Linear-embedding-based EEG Feature Reduction and Support-Vector-machine-based Clustering and Classification Techniques

Identifying Patients with Poststroke Mild Cognitive Impairment by Pattern Recognition of Working Memory Load-Related ERP

Enhanced Accuracy for Multiclass Mental Workload Detection Using Long Short-Term Memory for Brain–Computer Interface

Predicting Visual Working Memory with Multimodal Magnetic Resonance Imaging

Classification Algorithm for fNIRS-based Brain Signals Using Convolutional Neural Network with Spatiotemporal Feature Extraction Mechanism

[A deep transfer learning approach for cross-subject recognition of mental tasks based on functional near-infrared spectroscopy]

Topological EEG-Based Functional Connectivity Analysis for Mental Workload State Recognition

A novel AI-driven EEG generalized classification model for cross-subject and cross-scene analysis

Decoding Behavior Tasks From Brain Activity Using Deep Transfer Learning

Identifying Mental Workload Using EEG and Deep Learning

Deep Learning Decoding of Mental State in Non-invasive Brain Computer Interface

Decoding and mapping task states of the human brain via deep learning

Deep long short-term memory structures model temporal dependencies improving cognitive workload estimation

Decoding Working-Memory Load During n-Back Task Performance from High Channel NIRS Data