Abstract:Abstract Introduction Sleep disorders and deprivation disrupt people's daily activities, mental health, and longevity and are related to widespread conditions. Currently, sleep disorders are diagnosed via polysomnography (PSG), where electrophysiological data is collected and manually annotated by a clinician. State of the art machine learning (ML) models, such as the transformer, are particularly well-suited for modeling timeseries PSG data. Specifically, self-supervised models with linear probing could assist with any relevant sleep predictive task including automating sleep stage classification, which would save clinicians time, reduce variability in manual scoring, and help scale to treat more people. Methods Using a self-supervised learning approach and the transformer architecture, we trained a self-supervised ML model that inputs seven PSG channels of length three hours including electroencephalogram, electrooculogram, electromyography, electrocardiography, oxygen saturation, and thoracic and abdomen respiratory rate using the Sleep Heart Health Study database (1995-1998). The model architecture uses the transformer’s attention mechanism to learn long range dependencies between intervals of sleep and a convolutional layer to learn relationships among channels. The model learns representations of PSG data through masked reconstruction with a mean squared error loss function. The representations are used as input into a deep neural network that is trained via linear probing (without adjusting the weights of the transformer model) to classify sleep stages. Results 5,794 sleep studies from the Sleep Heart Health Study with at least three hours of relevant PSG sleep channel and hypnogram data are included in training the self-supervised and linear probing models. Area under the receiver operator characters curve for sleep stage classification are 0.960 [0.960-0.961], 0.848 [0.846-0.849], 0.906 [0.906-0.907], 0.968 [0.967-0.968], and 0.931 [0.930-0.932] for wake, stage 1, stage 2, stage 3, and REM, respectively. Hyperparameter tuning, class weighting, and dataset cleaning will be performed to increase classification results. Conclusion A self-supervised training approach using the transformer architecture with linear probing was utilized to learn multichannel PSG data representations. These representations were used as input into a downstream model to classify sleep stages accurately. Future work should be done to examine the capabilities of the self-supervised model representations for other predictive sleep tasks. Support (if any) NIH K25HL151912, NIH R01HL171813, NIH R21HL165320

A novel deep learning model based on transformer and cross modality attention for classification of sleep stages

Towards Interpretable Sleep Stage Classification Using Cross-Modal Transformers

Toward Interpretable Sleep Stage Classification Using Cross-Modal Transformers

CareSleepNet: A Hybrid Deep Learning Network for Automatic Sleep Staging

Transformer-Based Contrastive Learning Method for Automated Sleep Stages Classification

MultiChannelSleepNet: A Transformer-Based Model for Automatic Sleep Stage Classification with PSG

A CNN-Transformer-ConvLSTM-CRF Hybrid Network for Sleep Stage Classification

0320 Foundational Transformers with Linear Probing for Sleep Stage Classification Using Time Series Sleep Study Data

Classification of Infant Sleep/Wake States: Cross-Attention among Large Scale Pretrained Transformer Networks using Audio, ECG, and IMU Data

An Attention-Based Deep Learning Approach for Sleep Stage Classification With Single-Channel EEG

A convolutional transformer for EEG sleep stage classification

A Spatial-Temporal Transformer Architecture Using Multi-Channel Signals for Sleep Stage Classification

A foundational transformer leveraging full night, multichannel sleep study data accurately classifies sleep stages

Multi-Modal Sleep Stage Classification With Two-Stream Encoder-Decoder

A Multi Constrained Transformer-BiLSTM Guided Network for Automated Sleep Stage Classification from Single-Channel EEG

A CNN-Transformer Deep Learning Model for Real-time Sleep Stage Classification in an Energy-Constrained Wireless Device

Automatic Classification of Sleep Stages from EEG Signals Using Riemannian Metrics and Transformer Networks

Deep Neural Network method for Sleep Stages Classification using spectrogram of signal based on transfer learning with different domain data

0059 A Transformer Model for Predicting Cognitive Impairment from Sleep

A Hybrid Self-Attention Deep Learning Framework for Multivariate Sleep Stage Classification

A Hybrid Deep Learning Scheme for Multi-Channel Sleep Stage Classification