Explainable Depression Symptom Detection in Social Media

Eliseo Bao,Anxo Pérez,Javier Parapar

2024-08-20

Abstract:Users of social platforms often perceive these sites as supportive spaces to post about their mental health issues. Those conversations contain important traces about individuals' health risks. Recently, researchers have exploited this online information to construct mental health detection models, which aim to identify users at risk on platforms like Twitter, Reddit or Facebook. Most of these models are centred on achieving good classification results, ignoring the explainability and interpretability of the decisions. Recent research has pointed out the importance of using clinical markers, such as the use of symptoms, to improve trust in the computational models by health professionals. In this paper, we propose using transformer-based architectures to detect and explain the appearance of depressive symptom markers in the users' writings. We present two approaches: i) train a model to classify, and another one to explain the classifier's decision separately and ii) unify the two tasks simultaneously using a single model. Additionally, for this latter manner, we also investigated the performance of recent conversational LLMs when using in-context learning. Our natural language explanations enable clinicians to interpret the models' decisions based on validated symptoms, enhancing trust in the automated process. We evaluate our approach using recent symptom-based datasets, employing both offline and expert-in-the-loop metrics to assess the quality of the explanations generated by our models. The experimental results show that it is possible to achieve good classification results while generating interpretable symptom-based explanations.

Computation and Language

What problem does this paper attempt to address?

The main problem that this paper attempts to solve is to improve the interpretability of depression - symptom - detection models on social platforms. Specifically, the authors focus on:

1. **Accurate Classification and Explanation**: Develop models that can accurately identify whether there are markers of depression symptoms in social media posts and can provide natural - language explanations for these classification decisions. This includes not only identifying the presence or absence of depression symptoms but also why the model made such a judgment.

2. **Balance between Model Performance and Explanatory Power**: Explore how to generate reliable, symptom - based explanations while maintaining good classification performance. This involves studying the performance of different model architectures (such as one - step models and two - step models) as well as recent large language models (LLMs) on this task.

3. **Feasibility of Clinical Applications**: Ensure that the generated explanations conform to validated symptom criteria, so that clinicians can more effectively understand and trust these decisions. This requires that the model be not only technically effective but also useful to professionals in practical applications.

### Main Contributions of the Paper

- **Model Design**: Two methods are proposed to achieve the detection and explanation of depression symptoms: one is to train one model for classification and another model to separately explain the classification decisions; the other is to unify these two tasks in one model to complete them simultaneously.
- **Dataset Usage**: Utilize two symptom - focused datasets (PsySym and BDI - Sen) to evaluate the performance of the models through these datasets.
- **Evaluation Methods**: Employ off - line metrics and expert - involved evaluation methods to comprehensively evaluate the quality of model explanations.
- **Experimental Results**: Demonstrate that by giving priority to interpretability, good classification results can be achieved while generating interpretable symptom information, thus promoting more informed clinical decisions.

### Problems Solved

- **Lack of Interpretability**: Although existing models perform well in classification accuracy, they lack sufficient interpretability, which limits their application in clinical practice.
- **Model Generalization Ability**: By using clinical symptoms as markers, the generalization ability and overall performance of the model are improved.
- **Practicality and Credibility**: The generated explanations can be understood by clinicians, enhancing the credibility and practicality of the model in practical applications.

In conclusion, this paper aims to solve the interpretability and credibility problems in depression - symptom - detection by introducing new model architectures and evaluation methods, providing strong support for clinical applications.