Abstract:Parkinson's disease (PD) is a neurodegenerative disease which is prevalent among the elder population and severely affects the life quality of patients and their families. Therefore, it is important to conduct an early diagnosis for potential patients with PD, so as to promote prompt treatment and avoid the aggravation of the disease. Recently, the in-vitro PD diagnosis based on facial expressions has received increasing attention because of its distinguishability (i.e., PD patients always possess the characteristics of “masked face”) and affordability. However, the performance of the existing facial expression-based PD diagnosis approaches is limited by: 1) the small-scale training data on PD patients' facial expressions, and 2) the weak prediction model. To address these two problems, we propose a new facial expression guided PD diagnosis method based on high-quality training data augmentation and deep neural network prediction. Specifically, the proposed method consists of three stages: Firstly, we synthesize virtual facial expression images with 6 basic emotions (i.e., anger, disgust, fear, happiness, sadness, and surprise) based on multi-domain adversarial learning to approximate the premorbid expressions of PD patients. Secondly, we introduce three facial image quality assessment (FIQA) criteria to measure the quality of these synthesized facial expression images and design a fusion screening strategy that shortlists the high-quality ones to augment the training data. Finally, we train a deep neural network prediction model based on the original and synthesized high-quality facial expression images for PD diagnosis. To show real-world impacts and evaluate the proposed method under different facial expressions, we also create a (currently largest) multiple facial expressions-based PD face dataset in collaboration with a hospital. Extensive experiments are performed to demonstrate the effectiveness of the multi-domain adversarial learning-based facial expression synthesis and the fusion screening strategy, particularly the superior performance of the proposed method for PD diagnosis.

Simulated multimodal deep facial diagnosis

ASDFace: Face-based Autism Diagnosis Via Heterogeneous Domain Adaptation

Facial Expression Guided Diagnosis of Parkinson's Disease Via High-Quality Data Augmentation

Multiple Genetic Syndromes Recognition Based on a Deep Learning Framework and Cross-Loss Training

Automated syndrome diagnosis by three-dimensional facial imaging

Diagnosis and Screening of Velocardiofacial Syndrome by Evaluating Facial Photographs Using a Deep Learning-Based Algorithm

Artificial intelligence-assisted dermatology diagnosis: From unimodal to multimodal

GestaltMML: Enhancing Rare Genetic Disease Diagnosis through Multimodal Machine Learning Combining Facial Images and Clinical Texts

Research and application of tongue and face diagnosis based on deep learning

MMDS: A Multimodal Medical Diagnosis System Integrating Image Analysis and Knowledge-based Departmental Consultation

Medical Diagnosis with Large Scale Multimodal Transformers: Leveraging Diverse Data for More Accurate Diagnosis

DeepGestalt - Identifying Rare Genetic Syndromes Using Deep Learning

Facial recognition for disease diagnosis using a deep learning convolutional neural network: a systematic review and meta-analysis

Analysis of 3D Facial Dysmorphology in Genetic Syndromes from Unconstrained 2D Photographs.

Automated Multi-Class Facial Syndrome Classification Using Transfer Learning Techniques

Unpaired Multimodal Facial Expression Recognition.

Multimodal Genotype and Phenotype Data Integration to Improve Partial Data-Based Longitudinal Prediction

TAMM: A Task-Adaptive Multi-Modal Fusion Network for Facial-Related Health Assessments on 3D Facial Images

Facial Recognition Intensity in Disease Diagnosis Using Automatic Facial Recognition

Identification of the Facial Features of Patients With Cancer: A Deep Learning–Based Pilot Study

An Automatic Multi-View Disease Detection System Via Collective Deep Region-based Feature Representation