Analysis and evaluation of explainable artificial intelligence on suicide risk assessment

Hao Tang,Aref Miri Rekavandi,Dharjinder Rooprai,Girish Dwivedi,Frank M. Sanfilippo,Farid Boussaid,Mohammed Bennamoun
DOI: https://doi.org/10.1038/s41598-024-53426-0
IF: 4.6
2024-03-15
Scientific Reports
Abstract:This study explores the effectiveness of Explainable Artificial Intelligence (XAI) for predicting suicide risk from medical tabular data. Given the common challenge of limited datasets in health-related Machine Learning (ML) applications, we use data augmentation in tandem with ML to enhance the identification of individuals at high risk of suicide. We use SHapley Additive exPlanations (SHAP) for XAI and traditional correlation analysis to rank feature importance, pinpointing primary factors influencing suicide risk and preventive measures. Experimental results show the Random Forest (RF) model is excelling in accuracy, F1 score, and AUC (>97% across metrics). According to SHAP, anger issues, depression, and social isolation emerge as top predictors of suicide risk, while individuals with high incomes, esteemed professions, and higher education present the lowest risk. Our findings underscore the effectiveness of ML and XAI in suicide risk assessment, offering valuable insights for psychiatrists and facilitating informed clinical decisions.
multidisciplinary sciences
What problem does this paper attempt to address?
The main problem that this paper attempts to solve is to use Explainable Artificial Intelligence (XAI) techniques to improve the accuracy and interpretability of suicide risk prediction. Specifically, researchers predict suicide risk from medical form data by using Machine Learning (ML) methods, especially the Random Forest (RF) model, combined with data augmentation techniques. In addition, they also use the SHapley Additive exPlanations (SHAP) method to explain the prediction results of the model, determine the main factors affecting suicide risk, and evaluate the effectiveness of preventive measures. ### Background and Objectives of the Paper 1. **Background**: - Suicide is a global public health problem, causing more than 700,000 deaths every year, most of which occur in low - and middle - income countries. - Current suicide prevention tools mainly rely on self - reported questionnaires and interviews. These methods are highly subjective, difficult to collect, and lack accuracy. - Traditional clinical risk assessment tools perform poorly in identifying individuals at medium - to - high risk, and new technologies and models need to be developed to assist psychiatrists and mental health professionals in more accurate risk stratification. 2. **Objectives**: - **Literature Review**: Comprehensively review relevant literature to understand various Machine Learning models used for suicide prediction and their limited interpretability in clinical interventions. - **Model Selection and Data Augmentation**: Select appropriate Machine Learning algorithms and combine them with data augmentation techniques to evaluate the feasibility of these models in predicting suicidal tendencies. - **Variable Identification**: Use the Explainable Artificial Intelligence framework (XAI) to identify the most important variables affecting suicide risk and provide visual explanations behind the predictions. ### Main Findings - **Model Performance**: - The Random Forest (RF) model performs excellently in terms of accuracy, F1 - score, and AUC value, all exceeding 97%. - Data augmentation techniques significantly improve the performance of the model on multiple indicators. - **Key Variables**: - According to SHAP analysis, anger problems, depression, and social isolation are the most important factors in predicting suicide risk. - People with higher income, respected occupations, and higher education levels have the lowest suicide risk. ### Clinical Significance and Future Directions - **Clinical Significance**: - This study demonstrates the effectiveness of Machine Learning and Explainable Artificial Intelligence in suicide risk assessment, providing valuable insights for psychiatrists and helping them make more informed clinical decisions. - By combining Machine Learning models and clinical risk assessment tools, the accuracy of suicide risk prediction and the intervention effect can be improved. - **Future Directions**: - Develop a risk assessment interface, using the identified factors and Machine Learning algorithms to provide clinicians with individual suicide risk predictions. - Explore other modalities of data (such as voice, image, and video) to further improve the predictive ability and interpretability of the model. In conclusion, this paper provides a new and more accurate method for suicide risk assessment by combining Machine Learning and Explainable Artificial Intelligence techniques, which has important clinical application value.