A Within-Subject Multimodal NIRS-EEG Classifier for Infant Data

Gemignani,Gervain
DOI: https://doi.org/10.3390/s24134161
IF: 3.9
2024-06-27
Sensors
Abstract:Functional Near Infrared Spectroscopy (fNIRS) and Electroencephalography (EEG) are commonly employed neuroimaging methods in developmental neuroscience. Since they offer complementary strengths and their simultaneous recording is relatively easy, combining them is highly desirable. However, to date, very few infant studies have been conducted with NIRS-EEG, partly because analyzing and interpreting multimodal data is challenging. In this work, we propose a framework to carry out a multivariate pattern analysis that uses an NIRS-EEG feature matrix, obtained by selecting EEG trials presented within larger NIRS blocks, and combining the corresponding features. Importantly, this classifier is intended to be sensitive enough to apply to individual-level, and not group-level data. We tested the classifier on NIRS-EEG data acquired from five newborn infants who were listening to human speech and monkey vocalizations. We evaluated how accurately the model classified stimuli when applied to EEG data alone, NIRS data alone, or combined NIRS-EEG data. For three out of five infants, the classifier achieved high and statistically significant accuracy when using features from the NIRS data alone, but even higher accuracy when using combined EEG and NIRS data, particularly from both hemoglobin components. For the other two infants, accuracies were lower overall, but for one of them the highest accuracy was still achieved when using combined EEG and NIRS data with both hemoglobin components. We discuss how classification based on joint NIRS-EEG data could be modified to fit the needs of different experimental paradigms and needs.
engineering, electrical & electronic,chemistry, analytical,instruments & instrumentation
What problem does this paper attempt to address?
The problem this paper attempts to address is the development of a multimodal NIRS-EEG classifier to distinguish between the brain activity patterns of newborns in response to human speech and monkey calls at the individual subject level. Specifically, the study aims to: 1. **Overcome technical challenges**: Combine NIRS (functional near-infrared spectroscopy) and EEG (electroencephalography), two neuroimaging methods that have different advantages in spatial localization and temporal resolution, respectively. However, there are technical challenges in simultaneously recording and analyzing them. 2. **Improve classification accuracy**: Enhance the classification accuracy of newborn brain activity patterns through multimodal data fusion. The classification performance may be suboptimal when using NIRS or EEG data alone, but combining both can provide more comprehensive information, thereby improving classification performance. 3. **Validate feasibility at the individual level**: Perform classification at the individual level rather than the group level. This is particularly important in newborn research due to the significant differences between individuals, as well as variations in signal quality, number of trials, and maturity for each infant. 4. **Explore experimental design**: Employ an innovative experimental design that nests EEG trials within longer NIRS blocks to achieve direct temporal correlation of the two signals, thereby better leveraging the advantages of multimodal data. In summary, this paper aims to overcome the limitations of existing technologies, improve the classification accuracy of newborn brain activity patterns, and validate the feasibility of individual-level classification by proposing a multimodal NIRS-EEG classifier.