Abstract:Personality determines a wide variety of human daily and working behaviours, and is crucial for understanding human internal and external states. In recent years, a large number of automatic personality computing approaches have been developed to predict either the apparent personality or self-reported personality of the subject based on non-verbal audio-visual behaviours. However, the majority of them suffer from complex and dataset-specific pre-processing steps and model training tricks. In the absence of a standardized benchmark with consistent experimental settings, it is not only impossible to fairly compare the real performances of these personality computing models but also makes them difficult to be reproduced. In this paper, we present the first reproducible audio-visual benchmarking framework to provide a fair and consistent evaluation of eight existing personality computing models (e.g., audio, visual and audio-visual) and seven standard deep learning models on both self-reported and apparent personality recognition tasks. Building upon a set of benchmarked models, we also investigate the impact of two previously-used long-term modelling strategies for summarising short-term/frame-level predictions on personality computing results. We conduct a comprehensive investigation into all the benchmarked models to demonstrate their capabilities in modelling personality traits on two publicly available datasets, audio-visual apparent personality (ChaLearn First Impression) and self-reported personality (UDIVA) datasets. The experimental results conclude: (i) apparent personality traits, inferred from facial behaviours by most benchmarked deep learning models, show more reliability than self-reported ones; (ii) visual models frequently achieved superior performances than audio models on personality recognition; (iii) non-verbal behaviours contribute differently in predicting different personality traits; and (iv) our reproduced personality computing models generally achieved worse performances than their original reported results. We make all the code and settings of this personality computing benchmark publicly available at https://github.com/liaorongfan/DeepPersonality.

Deep Bimodal Regression of Apparent Personality Traits from Short Video Sequences.

Deep Bimodal Regression For Apparent Personality Analysis

Personality Analysis from Online Short Video Platforms with Multi-domain Adaptation

In Your Eyes: Modality Disentangling for Personality Analysis in Short Video

Dynamic Facial Features in Positive-Emotional Speech for Identification of Depressive Tendencies

Enhancing Apparent Personality Trait Analysis with Cross-Modal Embeddings

Domain-specific Learning of Multi-scale Facial Dynamics for Apparent Personality Traits Prediction

Multimodal Video-based Apparent Personality Recognition Using Long Short-Term Memory and Convolutional Neural Networks

Integrating audio and visual modalities for multimodal personality trait recognition via hybrid deep learning

Deep Impression: Audiovisual Deep Residual Networks for Multimodal Apparent Personality Trait Recognition

Multimodal analysis of personality traits on videos of self-presentation and induced behavior

An Open-source Benchmark of Deep Learning Models for Audio-visual Apparent and Self-reported Personality Recognition

CR-Net: A Deep Classification-Regression Network for Multimodal Apparent Personality Analysis

PersEmoN: A Deep Network for Joint Analysis of Apparent Personality, Emotion and Their Relationship

Personality Assessment Based on Multimodal Attention Network Learning With Category-Based Mean Square Error

Learning Expression Features via Deep Residual Attention Networks for Facial Expression Recognition From Video Sequences

Identifying Big Five personality traits based on facial behavior analysis

Investigating Audio, Visual, and Text Fusion Methods for End-to-End Automatic Personality Prediction

Your Face Mirrors Your Deepest Beliefs-Predicting Personality and Morals through Facial Emotion Recognition

The Deeper, the Better: Analysis of Person Attributes Recognition

Deep learning-based personality recognition from text posts of online social networks