Abstract:Abstract Introduction Sleep disorders and deprivation disrupt people's daily activities, mental health, and longevity and are related to widespread conditions. Currently, sleep disorders are diagnosed via polysomnography (PSG), where electrophysiological data is collected and manually annotated by a clinician. State of the art machine learning (ML) models, such as the transformer, are particularly well-suited for modeling timeseries PSG data. Specifically, self-supervised models with linear probing could assist with any relevant sleep predictive task including automating sleep stage classification, which would save clinicians time, reduce variability in manual scoring, and help scale to treat more people. Methods Using a self-supervised learning approach and the transformer architecture, we trained a self-supervised ML model that inputs seven PSG channels of length three hours including electroencephalogram, electrooculogram, electromyography, electrocardiography, oxygen saturation, and thoracic and abdomen respiratory rate using the Sleep Heart Health Study database (1995-1998). The model architecture uses the transformer’s attention mechanism to learn long range dependencies between intervals of sleep and a convolutional layer to learn relationships among channels. The model learns representations of PSG data through masked reconstruction with a mean squared error loss function. The representations are used as input into a deep neural network that is trained via linear probing (without adjusting the weights of the transformer model) to classify sleep stages. Results 5,794 sleep studies from the Sleep Heart Health Study with at least three hours of relevant PSG sleep channel and hypnogram data are included in training the self-supervised and linear probing models. Area under the receiver operator characters curve for sleep stage classification are 0.960 [0.960-0.961], 0.848 [0.846-0.849], 0.906 [0.906-0.907], 0.968 [0.967-0.968], and 0.931 [0.930-0.932] for wake, stage 1, stage 2, stage 3, and REM, respectively. Hyperparameter tuning, class weighting, and dataset cleaning will be performed to increase classification results. Conclusion A self-supervised training approach using the transformer architecture with linear probing was utilized to learn multichannel PSG data representations. These representations were used as input into a downstream model to classify sleep stages accurately. Future work should be done to examine the capabilities of the self-supervised model representations for other predictive sleep tasks. Support (if any) NIH K25HL151912, NIH R01HL171813, NIH R21HL165320

Toward Interpretable Sleep Stage Classification Using Cross-Modal Transformers

Towards Interpretable Sleep Stage Classification Using Cross-Modal Transformers

A novel deep learning model based on transformer and cross modality attention for classification of sleep stages

Transformer-Based Contrastive Learning Method for Automated Sleep Stages Classification

0320 Foundational Transformers with Linear Probing for Sleep Stage Classification Using Time Series Sleep Study Data

A Spatial-Temporal Transformer Architecture Using Multi-Channel Signals for Sleep Stage Classification

CareSleepNet: A Hybrid Deep Learning Network for Automatic Sleep Staging

Automatic Classification of Sleep Stages from EEG Signals Using Riemannian Metrics and Transformer Networks

Convolutional Transformer with Domain Adversarial Learning for Multi-Channel Sleep Stage Classification

MultiChannelSleepNet: A Transformer-Based Model for Automatic Sleep Stage Classification with PSG

Transparency in Sleep Staging: Deep Learning Method for EEG Sleep Stage Classification with Model Interpretability

Classification of Infant Sleep/Wake States: Cross-Attention among Large Scale Pretrained Transformer Networks using Audio, ECG, and IMU Data

Laying the Foundation: Modern Transformers for Gold-Standard Sleep Analysis and Beyond

Multi-Modal Sleep Stage Classification With Two-Stream Encoder-Decoder

Deep Convolutional Network Method For Automatic Sleep Stage Classification Based On Neurophysiological Signals

An Interpretable Single-Channel EEG Sleep Staging Model Based on Prototype Matching and Multi-Task Learning

A Multi Constrained Transformer-BiLSTM Guided Network for Automated Sleep Stage Classification from Single-Channel EEG

A CNN-Transformer-ConvLSTM-CRF Hybrid Network for Sleep Stage Classification

Deep Convolutional Neural Networks for Interpretable Analysis of EEG Sleep Stage Scoring

Temporal convolutional networks and transformers for classifying the sleep stage in awake or asleep using pulse oximetry signals

A Hybrid Deep Learning Scheme for Multi-Channel Sleep Stage Classification