Deep Learning for Sleep Stages Classification: Modified Rectified Linear Unit Activation Function and Modified Orthogonal Weight Initialisation

Akriti Bhusal,Abeer Alsadoon,P.W.C. Prasad,Nada Alsalami,Tarik A. Rashid

DOI: https://doi.org/10.1007/s11042-022-12372-7

2022-02-19

Abstract:Background and Aim: Each stage of sleep can affect human health, and not getting enough sleep at any stage may lead to sleep disorder like parasomnia, apnea, insomnia, etc. Sleep-related diseases could be diagnosed using Convolutional Neural Network Classifier. However, this classifier has not been successfully implemented into sleep stage classification systems due to high complexity and low accuracy of classification. The aim of this research is to increase the accuracy and reduce the learning time of Convolutional Neural Network Classifier. Methodology: The proposed system used a modified Orthogonal Convolutional Neural Network and a modified Adam optimisation technique to improve the sleep stage classification accuracy and reduce the gradient saturation problem that occurs due to sigmoid activation function. The proposed system uses Leaky Rectified Linear Unit (ReLU) instead of sigmoid activation function as an activation function. Results: The proposed system called Enhanced Sleep Stage Classification system (ESSC) used six different databases for training and testing the proposed model on the different sleep stages. These databases are University College Dublin database (UCD), Beth Israel Deaconess Medical Center MIT database (MIT-BIH), Sleep European Data Format (EDF), Sleep EDF Extended, Montreal Archive of Sleep Studies (MASS), and Sleep Heart Health Study (SHHS). Our results show that the gradient saturation problem does not exist anymore. The modified Adam optimiser helps to reduce the noise which in turn result in faster convergence time. Conclusion: The convergence speed of ESSC is increased along with better classification accuracy compared to the state of art solution.

Signal Processing,Machine Learning

What problem does this paper attempt to address?

The main problems that this paper attempts to solve are to improve the accuracy of convolutional neural networks (CNNs) in sleep - stage classification and reduce the learning time. Specifically, the author points out some major problems in existing methods:

1. **High complexity**: Existing CNN classifiers encounter difficulties in implementation due to their complex structures, especially in applications on portable devices.
2. **Low accuracy**: Current CNN systems perform poorly in extracting rich and diverse features of signals, resulting in low classification accuracy.
3. **Gradient saturation problem**: Using the sigmoid activation function will lead to gradient saturation, making the weight update slow and thus affecting the classification accuracy.

To overcome these problems, the paper proposes an improved method, mainly including the following aspects:

- **Modified Orthogonal Convolutional Neural Network (OCNN)**: By using orthogonal initialization of weights, the learning speed of the model is improved and it can be run on portable devices.
- **Modified activation function**: Use Leaky ReLU instead of the sigmoid activation function to avoid the gradient saturation problem, accelerate the weight update speed, and thus improve the classification accuracy.
- **Modified Adam optimizer**: The Adam optimizer is introduced to reduce the noise during the training process and accelerate the convergence time.

These improvements aim to improve the accuracy and efficiency of sleep - stage classification, making it more suitable for practical applications, especially on portable and wearable devices.