Abstract:Examination of dysfunctional brain dynamics often needs to tackle the difficulties of analyzing neural data of multiple modalities as recordings from a single source (such as EEG) do not always suffice in characterizing the brain states of interest. A typical example is sleep scoring from Polysomnography (PSG) for evaluation of sleep disorder, which has long been an onerous task and constantly results in unstable performance. Despite numerous successes achieved by the deep learning technologies recently booming along this direction, a grand challenge still remains: how to enable a stable solution to characterize the sleep stages (brain states) with temporal correlations embedded in the multi-modal recordings mainly from the brain activities without the support of sufficient a priori knowledge of the subjects.Aiming at the challenge, this study develops a deep learning framework for multi-modal sleep scoring. The framework uses a “dual-CNN” to simultaneously process inputs of two forms, i.e., temporal or time–frequency. The inputs of each branch may either consist of organized epochs of multiple modalities or their time–frequency representations. The fused deep features of the original multi-modal dataset will then be derived. The correlation of the original consecutive epochs embedded in the deep features can be reinforced through an RNN layer to aid classification with the final results fine-tuned by a customized Markov chain model, which considers the temporal correlations embedded in the deep features to amortize the potential bias resulted from the imbalance of brain states represented by the features.A case study of sleep scoring has been performed against a public Sleep-EDF dataset. Experimental results indicate that 1) the multi-modal features generated by the dual-CNN model help in significantly improve the accuracy of neural data classification; 2) the overall framework outperforms the state-of-the-art counterparts with an accuracy of 84.9% achieved in sleep scoring.

Convolutional Multiple Instance Learning for Sleep Spindle Detection with Label Refinement

Insomnia Prediction Using Temporal Feature of Spindles

Classification and transfer learning of sleep spindles based on convolutional neural networks

Automated Sleep Spindle Detection with Mixed EEG Features

An Efficient Sleep Spindle Detection Algorithm Based on MP and LSBoost

SpindleU-Net: An Adaptive U-Net Framework for Sleep Spindle Detection in Single-Channel EEG

A robust two-stage sleep spindle detection approach using single-channel EEG

Hierarchical Fusion Detection Algorithm for Sleep Spindle Detection

Automatic Detection of Sleep Spindles and Its Application in Patients with Acute Disorders of Consciousness

Advanced sleep spindle identification with neural networks

Deep-spindle: an Automated Sleep Spindle Detection System for Analysis of Infant Sleep Spindles.

Complex-valued unsupervised convolutional neural networks for sleep stage classification

Spindle Detection Based on Elastic Time Window and Spatial Pyramid Pooling

Automatic Sleep Stage Classification Based on Convolutional Neural Network and Fine-Grained Segments.

Dual-CNN Based Multi-Modal Sleep Scoring with Temporal Correlation Driven Fine-Tuning

Sle-CNN: a novel convolutional neural network for sleep stage classification

Competition convolutional neural network for sleep stage classification

Computation-Efficient Multi-Model Deep Neural Network for Sleep Stage Classification.

Using a Deep CNN for Automatic Classification of Sleep Spindles: A Preliminary Study

Actigraphy-based Sleep/Wake Pattern Detection using Convolutional Neural Networks

A Novel Sleep Stage Contextual Refinement Algorithm Leveraging Conditional Random Fields