Personality prediction from task-oriented and open-domain human–machine dialogues

Ao Guo,Ryu Hirai,Atsumoto Ohashi,Yuya Chiba,Yuiko Tsunomori,Ryuichiro Higashinaka

DOI: https://doi.org/10.1038/s41598-024-53989-y

IF: 4.6

2024-02-18

Scientific Reports

Abstract:If a dialogue system can predict the personality of a user from dialogue, it will enable the system to adapt to the user's personality, leading to better task success and user satisfaction. In a recent study, personality prediction was performed using the Myers–Briggs Type Indicator (MBTI) personality traits with a task-oriented human–machine dialogue using an end-to-end (neural-based) system. However, it is still not clear whether such prediction is generally possible for other types of systems and user personality traits. To clarify this, we recruited 378 participants, asked them to fill out four personality questionnaires covering 25 personality traits, and had them perform three rounds of human–machine dialogue with a pipeline task-oriented dialogue system or an end-to-end task-oriented dialogue system. We also had another 186 participants do the same with an open-domain dialogue system. We then constructed BERT-based models to predict the personality traits of the participants from the dialogues. The results showed that prediction accuracy was generally better with open-domain dialogue than with task-oriented dialogue, although Extraversion (one of the Big Five personality traits) could be predicted equally well for both open-domain dialogue and pipeline task-oriented dialogue. We also examined the effect of utilizing different types of dialogue on personality prediction by conducting a cross-comparison of the models trained from the task-oriented and open-domain dialogues. As a result, we clarified that the open-domain dialogue cannot be used to predict personality traits from task-oriented dialogue, and vice versa. We further analyzed the effects of system utterances, task performance, and the round of dialogue with regard to the prediction accuracy.

multidisciplinary sciences

What problem does this paper attempt to address?

The problems that this paper attempts to solve are: whether it is feasible to predict users' personality traits from task - oriented dialogues and open - domain dialogues, and to explore the influence of different types of dialogue systems on personality prediction. Specifically, the researchers hope to understand:

1. **Feasibility of personality prediction**: Whether the personality traits of users can be effectively predicted through the dialogue system, especially the Big Five personality traits, IOS (Inclusion of Other in the Self), 25 personality traits in KISS - 18 (Kikuchi’s Scale of Social Skills) and ATQ (Adult Temperament Questionnaire).
2. **Effects of different types of dialogues**: Compare the differences in the effects of task - oriented dialogues (including pipeline - style and end - to - end systems) and open - domain dialogues on personality prediction.
3. **Generality of cross - dialogue - type prediction**: Whether a model trained on one type of dialogue can be applied to personality prediction of another type of dialogue.
4. **Analysis of influencing factors**: Explore the influence of factors such as system utterances, task performance and dialogue turns on the accuracy of personality prediction.

The researchers collected a large amount of dialogue data of users in different dialogue systems through a large - scale experimental design, and used BERT - based models for personality prediction to verify the above problems.