Abstract:Depression is a serious mental disorder, significantly burdens individuals, families, and society. For clinical psychiatrists, assessing the severity of depression is a crucial tool in selecting treatment approaches and evaluating their effectiveness. Although many studies in machine learning have focused on the automatic evaluation of self-rating scales [such as the Beck Depression Inventory-II (BDI-II) and the Patient Health Questionnaire-8 (PHQ-8)], research into the machine learning-based automatic evaluation of the medical clinical assessment scale [such as the Hamilton Depression Scale (HAMD)] has not yet been focused on. In this study, an end-to-end automatic evaluation device for HAMD and Patient Health Questionnaire-9 (PHQ-9) scores was developed. In addition, we constructed a dataset consisting of emotional facial expression videos (eFEVs) signals and emotional pupil-wave (ePW) signals from 65 patients with depression. The dataset has HAMD and PHQ-9 score labels, encompassing two emotional states: sadness and happiness. We built a 3-dimensional convolutional neural network + long short-term memory (3DCNN + LSTM) model framework and a multiscale 1-dimensional convolutional neural network (1DCNN) to learn and extract features from eFEVs and ePW automatically. The results showed that compared with the previous evaluation methods for depression levels, the evaluation precision of HAMD and PHQ-9 has been improved significantly. The results also showed that, in both HAMD and PHQ-9 evaluations, the evaluation precision of eFEVs was superior to ePW, and HAMD is better than PHQ-9. These studies indicated that both emotional facial expressions and ePW can better represent depressive mood in patients with depression, especially emotional facial expressions, and the predictive precision of the medical scale is significantly better than the self-rating scale. This automated assessment method and device can assist doctors in diagnosing depressive symptoms more effectively and serve as an evaluation tool for treatment efficacy.

Psychological test of patients with mental disorders based on eye movement data fusion algorithm

Eye Blink Artifact Detection Based on Multi-Dimensional EEG Feature Fusion and Optimization.

Automatic Emotion Perception Using Eye Movement Information for E-Healthcare Systems

Multimodal Emotion Recognition from Eye Image, Eye Movement and EEG Using Deep Neural Networks

Development of an eye-tracking system based on a deep learning model to assess executive function in patients with mental illnesses

Method of Depression Classification Based on Behavioral and Physiological Signals of Eye Movement.

Research on Data Fusion Algorithm Based on Deep Learning in Target Tracking

A novel deep learning approach for diagnosing Alzheimer's disease based on eye-tracking data

Active Feedback Framework with Scan-Path Clustering for Deep Affective Models

EEG-Based Depression Recognition Using Convolutional Neural Network with FFT and EMD

Multimodal Mild Depression Recognition Based On Eeg-Em Synchronization Acauisition Network

Multimodal Mild Depression Recognition Based on EEG-EM Synchronization Acquisition Network

EEG-based Emotion Recognition with Feature Fusion Networks.

STM-Net Based Spatial-Temporal Multi-Modal Fusion Network for Emotion Recognition

Content-based multiple evidence fusion on EEG and eye movements for mild depression recognition

EMS: A Large-Scale Eye Movement Dataset, Benchmark, and New Model for Schizophrenia Recognition

EmMixformer: Mix transformer for eye movement recognition

Automatic Assessment Method and Device for Depression Symptom Severity Based on Emotional Facial Expression and Pupil-Wave

Fusing of Electroencephalogram and Eye Movement with Group Sparse Canonical Correlation Analysis for Anxiety Detection

HASTF: a hybrid attention spatio-temporal feature fusion network for EEG emotion recognition

Residual self-calibrated network with multi-scale channel attention for accurate EOG-based eye movement classification.