Abstract:Personality determines a wide variety of human daily and working behaviours, and is crucial for understanding human internal and external states. In recent years, a large number of automatic personality computing approaches have been developed to predict either the apparent personality or self-reported personality of the subject based on non-verbal audio-visual behaviours. However, the majority of them suffer from complex and dataset-specific pre-processing steps and model training tricks. In the absence of a standardized benchmark with consistent experimental settings, it is not only impossible to fairly compare the real performances of these personality computing models but also makes them difficult to be reproduced. In this paper, we present the first reproducible audio-visual benchmarking framework to provide a fair and consistent evaluation of eight existing personality computing models (e.g., audio, visual and audio-visual) and seven standard deep learning models on both self-reported and apparent personality recognition tasks. Building upon a set of benchmarked models, we also investigate the impact of two previously-used long-term modelling strategies for summarising short-term/frame-level predictions on personality computing results. We conduct a comprehensive investigation into all the benchmarked models to demonstrate their capabilities in modelling personality traits on two publicly available datasets, audio-visual apparent personality (ChaLearn First Impression) and self-reported personality (UDIVA) datasets. The experimental results conclude: (i) apparent personality traits, inferred from facial behaviours by most benchmarked deep learning models, show more reliability than self-reported ones; (ii) visual models frequently achieved superior performances than audio models on personality recognition; (iii) non-verbal behaviours contribute differently in predicting different personality traits; and (iv) our reproduced personality computing models generally achieved worse performances than their original reported results. We make all the code and settings of this personality computing benchmark publicly available at https://github.com/liaorongfan/DeepPersonality.

First Impression: AI Understands Personality

Better Than Humans: a Method for Inferring Consumer Shopping Intentions by Reading Facial Expressions

A real-time interactive system for facial makeup of Peking Opera

Analysis and Synthesis of Peking Opera Facial Make-Ups

3D Face Synthesis Driven by Personality Impression

Explaining First Impressions: Modeling, Recognizing, and Explaining Apparent Personality from Videos

How Your Portrait Impresses People?

A deepfake-based study on facial expressiveness and social outcomes

An Open-source Benchmark of Deep Learning Models for Audio-visual Apparent and Self-reported Personality Recognition

Comparison of face‐based and voice‐based first impressions in a Chinese sample

Automated Inference on Sociopsychological Impressions of Attractive Female Faces

Don't judge a book by its cover: First impressions from faces are consensual but inaccurate

Your Face Mirrors Your Deepest Beliefs-Predicting Personality and Morals through Facial Emotion Recognition

The differences in essential facial areas for impressions between humans and deep learning models: An eye‐tracking and explainable AI approach

Thinking Like Psychologist: Learning to Predict Personality by Using Features from Portrait and Social Media.

Integrating audio and visual modalities for multimodal personality trait recognition via hybrid deep learning

Exploring Relationship Between Face and Trustworthy Impression Using Mid-level Facial Features.

ExpressionBot: An Emotive Lifelike Robotic Face for Face-to-Face Communication

Learning to see people like people

Bio-Inspired Deep Attribute Learning Towards Facial Aesthetic Prediction

The Human Facial Combined Expression Recognition System