Abstract:Background:Personality psychology studies personality and its variation among individuals and is an essential branch of psychology. In recent years, machine learning research related to personality assessment has started to focus on the online environment and showed outstanding performance in personality assessment. However, the aspects of the personality of these prediction models measure remain unclear because few studies focus on the interpretability of personality prediction models. The objective of this study is to develop and validate a machine learning model with domain knowledge introduced to enhance accuracy and improve interpretability.Methods:Study participants were recruited via an online experiment platform. After excluding unqualified participants and downloading the Weibo posts of eligible participants, we used six psycholinguistic and mental health-related lexicons to extract textual features. Then the predictive personality model was developed using the multi-objective extra trees method based on 3,411 pairs of social media expression and personality trait scores. Subsequently, the prediction model's validity and reliability were evaluated, and each lexicon's feature importance was calculated. Finally, the interpretability of the machine learning model was discussed.Results:The features from Culture Value Dictionary were found to be the most important predictors. The fivefold cross-validation results regarding the prediction model for personality traits ranged between 0.44 and 0.48 (p < 0.001). The correlation coefficients of five personality traits between the two "split-half" datasets data ranged from 0.84 to 0.88 (p < 0.001). Moreover, the model performed well in terms of contractual validity.Conclusion:By introducing domain knowledge to the development of a machine learning model, this study not only ensures the reliability and validity of the prediction model but also improves the interpretability of the machine learning method. The study helps explain aspects of personality measured by such prediction models and finds a link between personality and mental health. Our research also has positive implications regarding the combination of machine learning approaches and domain knowledge in the field of psychiatry and its applications to mental health.

Using Machine Learning to Advance Personality Assessment and Theory

The Potential of Machine Learning Methods in Psychological Assessment and Test Construction

Psychometric and Validity Issues in Machine Learning Approaches to Personality Assessment: A Focus on Social Media Text Mining

How Social Media Expression Can Reveal Personality.

Automatic personality assessment through social media language.

Digital data and personality: A systematic review and meta-analysis of human perception and computer prediction.

Scoring a forced-choice image-based assessment of personality: A comparison of machine learning, regression, and summative approaches

Evaluating and Inducing Personality in Pre-trained Language Models

Combining machine learning algorithms for personality trait prediction

Language-based personality: a new approach to personality in a digital world

Personality Predictions Based on User Behavior on the Facebook Social Media Platform

Using passive sensor data to probe associations of social structure with changes in personality: A synthesis of network analysis and machine learning

Personality Trait Inference Via Mobile Phone Sensors: A Machine Learning Approach

Developing and evaluating language-based machine learning algorithms for inferring applicant personality in video interviews

How much can personality predict prosocial behavior?

Personality Profiling: How informative are social media profiles in predicting personal information?

Prediction of MBTI Personality Leveraging Machine Learning Algorithms

Machine Mindset: An MBTI Exploration of Large Language Models

A deep learning approach to text-based personality prediction using multiple data sources mapping

Feasibility of virtual reality and machine learning to assess personality traits in an organizational environment