Abstract:Sleep staging serves as a fundamental assessment for sleep quality measurement and sleep disorder diagnosis. Although current deep learning approaches have successfully integrated multimodal sleep signals, enhancing the accuracy of automatic sleep staging, certain challenges remain, as follows: 1) optimizing the utilization of multi-modal information complementarity, 2) effectively extracting both long- and short-range temporal features of sleep information, and 3) addressing the class imbalance problem in sleep data. To address these challenges, this paper proposes a two-stream encode-decoder network, named TSEDSleepNet, which is inspired by the depth sensitive attention and automatic multi-modal fusion (DSA2F) framework. In TSEDSleepNet, a two-stream encoder is used to extract the multiscale features of electrooculogram (EOG) and electroencephalogram (EEG) signals. And a self-attention mechanism is utilized to fuse the multiscale features, generating multi-modal saliency features. Subsequently, the coarser-scale construction module (CSCM) is adopted to extract and construct multi-resolution features from the multiscale features and the salient features. Thereafter, a Transformer module is applied to capture both long- and short-range temporal features from the multi-resolution features. Finally, the long- and short-range temporal features are restored with low-layer details and mapped to the predicted classification results. Additionally, the Lovász loss function is applied to alleviate the class imbalance problem in sleep datasets. Our proposed method was tested on the Sleep-EDF-39 and Sleep-EDF-153 datasets, and it achieved classification accuracies of 88.9% and 85.2% and Macro-F1 scores of 84.8% and 79.7%, respectively, thus outperforming conventional traditional baseline models. These results highlight the efficacy of the proposed method in fusing multi-modal information. This method has potential for application as an adjunct tool for diagnosing sleep disorders.

MAGSleepNet: Adaptively multi-scale temporal focused sleep staging model for multi-age groups

CareSleepNet: A Hybrid Deep Learning Network for Automatic Sleep Staging

Sleep Staging Based on Multi Scale Dual Attention Network

A Semi-supervised Multi-scale Arbitrary Dilated Convolution Neural Network for Pediatric Sleep Staging

A Sequential End-to-End Neonatal Sleep Staging Model with Squeeze and Excitation Blocks and Sequential Multi-scale Convolution Neural Networks

AFSleepNet: Attention-Based Multi-View Feature Fusion Framework for Pediatric Sleep Staging

MCASleepNet: Multimodal Channel Attention-Based Deep Neural Network for Automatic Sleep Staging

Multi-Modal Sleep Stage Classification With Two-Stream Encoder-Decoder

MFASleepNet: Multi-view Fusion Attention-Based Deep Neural Network for Automatic Sleep Staging

Dual-modal and Multi-Scale Deep Neural Networks for Sleep Staging Using EEG and ECG Signals.

LGSleepNet: an Automatic Sleep Staging Model Based on Local and Global Representation Learning.

A Temporal-Spectral Fused and Attention-based Deep Model for Automatic Sleep Staging

MaskSleepNet: A Cross-modality Adaptation Neural Network for Heterogeneous Signals Processing in Sleep Staging

A Multi-Class Automatic Sleep Staging Method Based on Long Short-Term Memory Network Using Single-Lead Electrocardiogram Signals

An End-to-End Sleep Staging Simulator Based on Mixed Deep Neural Networks.

A Novel Sleep Staging Network Based on Multi-Scale Dual Attention

Extracting Multi-Scale and Salient Features by MSE Based U-Structure and CBAM for Sleep Staging

Towards Real-Time Sleep Stage Prediction and Online Calibration Based on Architecturally Switchable Deep Learning Models

LWSleepNet: A lightweight attention-based deep learning model for sleep staging with singlechannel EEG

A Novel Sleep Staging Network Based on Data Adaptation and Multimodal Fusion.

Temporal Self-Attentional and Adaptive Graph Convolutional Mixed Model for Sleep Staging