Faces of the Mind: Unveiling Mental Health States Through Facial Expressions in 11,427 Adolescents

Xiao Xu,Keyin Zhou,Yan Zhang,Yang Wang,Fei Wang,Xizhe Zhang

2024-05-30

Abstract:Mood disorders, including depression and anxiety, often manifest through facial expressions. While previous research has explored the connection between facial features and emotions, machine learning algorithms for estimating mood disorder severity have been hindered by small datasets and limited real-world application. To address this gap, we analyzed facial videos of 11,427 participants, a dataset two orders of magnitude larger than previous studies. This comprehensive collection includes standardized facial expression videos from reading tasks, along with a detailed psychological scale that measures depression, anxiety, and stress. By examining the relationships among these emotional states and employing clustering analysis, we identified distinct subgroups embodying different emotional profiles. We then trained tree-based classifiers and deep learning models to estimate emotional states from facial features. Results indicate that models previously effective on small datasets experienced decreased performance when applied to our large dataset, highlighting the importance of data scale and mitigating overfitting in practical settings. Notably, our study identified subtle shifts in pupil dynamics and gaze orientation as potential markers of mood disorders, providing valuable information on the interaction between facial expressions and mental health. This research marks the first large-scale and comprehensive investigation of facial expressions in the context of mental health, laying the groundwork for future data-driven advancements in this field.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The problem that this paper attempts to solve is to evaluate the mental health status of adolescents through facial expression analysis, especially depression, anxiety and stress levels. Specifically, the research aims to:

1. **Construct a large - scale benchmark dataset**: Create a high - quality facial video dataset (FACES) containing 11,427 participants for evaluating the mental health status of adolescents. This dataset is two orders of magnitude larger than existing datasets, which helps to improve the generalization ability of machine - learning models and reduce over - fitting.

2. **Identify emotional subgroups**: Through cluster analysis, identify different emotional subgroups that represent different combinations of emotional characteristics. This helps to understand the complexity of mental health status more subtly, rather than just a single symptom manifestation.

3. **Develop an automatic detection method**: Use tree - based classifiers and deep - learning models to estimate emotional states from facial features. The research found that models that performed well on small datasets previously had a decline in performance on large - scale datasets, emphasizing the importance of data scale and preventing over - fitting.

4. **Explore key emotional indicators**: The research found that subtle changes in pupil dynamics and gaze direction may be potential signs of mood disorders, providing valuable information for understanding the relationship between facial expressions and mental health.

Through these efforts, the research aims to lay the foundation for future data - based adolescent mental health research and clinical applications, and provide more accurate, reliable and efficient methods for evaluating mood disorders.