Abstract:Objective Facial images have been shown to convey mental conditions as clinical symptoms. This study aimed to use facial images to detect patients with drug-naive schizophrenia (DN-SCZ) or chronic schizophrenia (C-SCZ) from healthy controls (HCs), and to investigate differences in facial expressions among these 3 groups, as well as relationships between facial expressions and psychiatric symptoms. Methods We recruited 45 DN-SCZ patients, 106 C-SCZ patients and 101 HCs for the study, and videotaped their facial expressions through a fixed experimental paradigm. The video data was converted to facial images and divided into two sets: one for training a group classification- convolutional neural network (CNN) with the classification of DN-SCZ patient, C-SCZ patient and HC as output, and the other for evaluating classification results of the group classification-CNN. Subsequently, we extracted and evaluated 300 labeled facial images for each basic facial expression. These labeled images were employed to train separate facial expression-CNNs for each group (DN-SCZ, C-SCZ, and HCs). All facial images from the videos were then processed by their facial expression-CNNs to output the most probable facial expressions. The psychiatric symptoms were assessed using the Positive and Negative Syndrome Scale. Statistical analyses were conducted on the predicted facial expressions to identify differences among the groups, and to examine relationships between the predicted facial expressions and the clinical data of DN/C-SCZ patients. Results The group classification-CNN achieved an accuracy of 90.99% in correctly classifying participants based on facial images. The 3 facial expression-CNNs achieved accuracies of 95.95%, 87.23%, and 92.11% in predicting 8 basic facial expressions within the 3 groups. Facial images of HCs were rated higher in valence, arousal and attractiveness, but lower in deviation from normal face than those of DN/C-SCZ patients. Happy images of DN-SCZ patients were rated lower in valence and arousal than those of C-SCZ patients, while their angry images were rated higher in arousal, attractiveness and deviation from normal images than those of C-SCZ patients. Within the fixed experimental paradigm, DN-SCZ patients exhibited sadder, more surprised expressions, while displaying fewer happy, angry and disgusted expressions, statistical metrics of their fearful and angry expressions were correlated with their total positive symptom score and total general psychopathology score, respectively. C-SCZ patients exhibited happier, more content, angry and neutral expressions, while showing fewer surprised expressions, no significant relationships were observed between their facial expressions and clinical data. Conclusions Facial expressions can potentially serve as indicative signs for detecting DN-SCZ and C-SCZ patients. There are objective differences in certain facial expressions among the 3 groups, and certain facial expressions in DN-SCZ patients are associated with some of their psychiatric symptoms.

Application of convolutional neural networks in the diagnosis of schizophrenia

3D convolutional neural network for schizophrenia detection using as EEG-based functional brain network

Automatic recognition of schizophrenia from brain-network features using graph convolutional neural network

Automated diagnosis of schizophrenia based on spatial–temporal residual graph convolutional network

Classification of schizophrenia patients using a graph convolutional network: A combined functional MRI and connectomics analysis

Discriminative analysis of schizophrenia patients using an integrated model combining 3D CNN with 2D CNN: A multimodal MR image and connectomics analysis

Automatic Diagnosis of Schizophrenia in EEG Signals Using CNN-LSTM Models

Automated Rest EEG-Based Diagnosis of Depression and Schizophrenia Using a Deep Convolutional Neural Network

EEG-based schizophrenia detection using fusion of effective connectivity maps and convolutional neural networks with transfer learning

A convolutional neural network-based diagnostic method using resting-state electroencephalograph signals for major depressive and bipolar disorders

A deep learning approach for diagnosis of schizophrenia disorder via data augmentation based on convolutional neural network and long short-term memory

Automatic identification of schizophrenia based on EEG signals using dynamic functional connectivity analysis and 3D convolutional neural network

Early Diagnosis of Alzheimer's Disease Using Deep Learning

Sparsity‐guided multiple functional connectivity patterns for classification of schizophrenia via convolutional network

Assisting Schizophrenia Diagnosis Using Clinical Electroencephalography and Interpretable Graph Neural Networks: a Real-World and Cross-Site Study

Facial expression analysis using convolutional neural network for drug-naive and chronic schizophrenia

A hybrid deep neural network for classification of schizophrenia using EEG Data

Discriminative analysis of schizophrenia patients using graph convolutional networks: A combined multimodal MRI and connectomics analysis

Multi feature fusion network for schizophrenia classification and abnormal brain network recognition

Machine learning techniques for the Schizophrenia diagnosis: a comprehensive review and future research directions

Detecting schizophrenia with 3D structural brain MRI using deep learning