Abstract:Personality analysis from online short videos has gained prominence due to its applications in personalized recommendation systems, sentiment analysis, and human-computer interaction. Traditional assessment methods, such as questionnaires based on the Big Five Personality Framework, are limited by self-report biases and are impractical for large-scale or real-time analysis. Leveraging the rich, multi-modal data present in short videos offers a promising alternative for more accurate personality inference. However, integrating these diverse and asynchronous modalities poses significant challenges, particularly in aligning time-varying data and ensuring models generalize well to new domains with limited labeled data. In this paper, we propose a novel multi-modal personality analysis framework that addresses these challenges by synchronizing and integrating features from multiple modalities and enhancing model generalization through domain adaptation. We introduce a timestamp-based modality alignment mechanism that synchronizes data based on spoken word timestamps, ensuring accurate correspondence across modalities and facilitating effective feature integration. To capture temporal dependencies and inter-modal interactions, we employ Bidirectional Long Short-Term Memory networks and self-attention mechanisms, allowing the model to focus on the most informative features for personality prediction. Furthermore, we develop a gradient-based domain adaptation method that transfers knowledge from multiple source domains to improve performance in target domains with scarce labeled data. Extensive experiments on real-world datasets demonstrate that our framework significantly outperforms existing methods in personality prediction tasks, highlighting its effectiveness in capturing complex behavioral cues and robustness in adapting to new domains.

Cognition-Driven Real-Time Personality Detection Via Language-Guided Contrastive Visual Attention.

Cognition-driven multimodal personality classification

Learning Person-specific Cognition from Facial Reactions for Automatic Personality Recognition

Personality Assessment Based on Multimodal Attention Network Learning With Category-Based Mean Square Error

Personality-aware Human-centric Multimodal Reasoning: A New Task, Dataset and Baselines

EERPD: Leveraging Emotion and Emotion Regulation for Improving Personality Detection

PsyCoT: Psychological Questionnaire as Powerful Chain-of-Thought for Personality Detection

2.5D Facial Personality Prediction Based on Deep Learning

Real-Time Eye-Gaze Based Interaction for Human Intention Prediction and Emotion Analysis.

A real-time attitude recognition by eye-tracking

Integrating audio and visual modalities for multimodal personality trait recognition via hybrid deep learning

Learning Graph Representation of Person-specific Cognitive Processes from Audio-visual Behaviours for Automatic Personality Recognition

Personality Driven Multi-task Learning for Image Aesthetic Assessment

Affective-NLI: Towards Accurate and Interpretable Personality Recognition in Conversation

Video-based multimodal personality analysis

PersonalitySensing: A Multi-View Multi-Task Learning Approach for Personality Detection Based on Smartphone Usage

Learning to Answer Psychological Questionnaire for Personality Detection

Data Augmented Graph Neural Networks for Personality Detection

Multimodal human attention detection for reading from facial expression, eye gaze, and mouse dynamics

PsyAttention: Psychological Attention Model for Personality Detection

Personality Analysis from Online Short Video Platforms with Multi-domain Adaptation