Abstract:Abstract Introduction Sleep disorders and deprivation disrupt people's daily activities, mental health, and longevity and are related to widespread conditions. Currently, sleep disorders are diagnosed via polysomnography (PSG), where electrophysiological data is collected and manually annotated by a clinician. State of the art machine learning (ML) models, such as the transformer, are particularly well-suited for modeling timeseries PSG data. Specifically, self-supervised models with linear probing could assist with any relevant sleep predictive task including automating sleep stage classification, which would save clinicians time, reduce variability in manual scoring, and help scale to treat more people. Methods Using a self-supervised learning approach and the transformer architecture, we trained a self-supervised ML model that inputs seven PSG channels of length three hours including electroencephalogram, electrooculogram, electromyography, electrocardiography, oxygen saturation, and thoracic and abdomen respiratory rate using the Sleep Heart Health Study database (1995-1998). The model architecture uses the transformer’s attention mechanism to learn long range dependencies between intervals of sleep and a convolutional layer to learn relationships among channels. The model learns representations of PSG data through masked reconstruction with a mean squared error loss function. The representations are used as input into a deep neural network that is trained via linear probing (without adjusting the weights of the transformer model) to classify sleep stages. Results 5,794 sleep studies from the Sleep Heart Health Study with at least three hours of relevant PSG sleep channel and hypnogram data are included in training the self-supervised and linear probing models. Area under the receiver operator characters curve for sleep stage classification are 0.960 [0.960-0.961], 0.848 [0.846-0.849], 0.906 [0.906-0.907], 0.968 [0.967-0.968], and 0.931 [0.930-0.932] for wake, stage 1, stage 2, stage 3, and REM, respectively. Hyperparameter tuning, class weighting, and dataset cleaning will be performed to increase classification results. Conclusion A self-supervised training approach using the transformer architecture with linear probing was utilized to learn multichannel PSG data representations. These representations were used as input into a downstream model to classify sleep stages accurately. Future work should be done to examine the capabilities of the self-supervised model representations for other predictive sleep tasks. Support (if any) NIH K25HL151912, NIH R01HL171813, NIH R21HL165320

Transformer-Based Contrastive Learning Method for Automated Sleep Stages Classification

Automatic Classification of Sleep Stages Based on Raw Single-Channel EEG

CareSleepNet: A Hybrid Deep Learning Network for Automatic Sleep Staging

MultiChannelSleepNet: A Transformer-Based Model for Automatic Sleep Stage Classification with PSG

Convolutional Transformer with Domain Adversarial Learning for Multi-Channel Sleep Stage Classification

Expert Knowledge Inspired Contrastive Learning for Sleep Staging.

FlexSleepTransformer: a transformer-based sleep staging model with flexible input channel configurations

A CNN-Transformer-ConvLSTM-CRF Hybrid Network for Sleep Stage Classification

Automatic Classification of Sleep Stages from EEG Signals Using Riemannian Metrics and Transformer Networks

Convolution- and Attention-Based Neural Network for Automated Sleep Stage Classification

Towards Interpretable Sleep Stage Classification Using Cross-Modal Transformers

Deep Convolutional Network Method For Automatic Sleep Stage Classification Based On Neurophysiological Signals

0320 Foundational Transformers with Linear Probing for Sleep Stage Classification Using Time Series Sleep Study Data

Toward Interpretable Sleep Stage Classification Using Cross-Modal Transformers

A Spatial-Temporal Transformer Architecture Using Multi-Channel Signals for Sleep Stage Classification

Automatic Sleep Staging Method Based on Multi-Channel Fusion

A convolutional transformer for EEG sleep stage classification

MSF-SleepNet: Multi-Stream Fusion Network with Contrastive Learning for Sleep Stage Classification.

Time-Frequency Convolutional Neural Network for Automatic Sleep Stage Classification Based on Single-Channel EEG

Self-supervised Contrastive Learning for EEG-based Sleep Staging

An Interpretable Single-Channel EEG Sleep Staging Model Based on Prototype Matching and Multi-Task Learning