Abstract:This paper proposes an adaptive multimodal information fusion network for personality recognition. The design features of each encoder are optimized and merged for downstream tasks. We greatly enhance the functionality of the Transformer component by integrating adaptive attention and automatic learning of cross‐modal associations. This not only solves the problem of outliers and gradient vanishing during model training, but also has practical significance for practical applications. Personality recognition is of great significance in deepening the understanding of social relations. While personality recognition methods have made significant strides in recent years, the challenge of heterogeneity between modalities during feature fusion still needs to be solved. This paper introduces an adaptive multi‐modal information fusion network (AMIF‐Net) capable of concurrently processing video, audio, and text data. First, utilizing the AMIF‐Net encoder, we process the extracted audio and video features separately, effectively capturing long‐term data relationships. Then, adding adaptive elements in the fusion network can alleviate the problem of heterogeneity between modes. Lastly, we concatenate audio‐video and text features into a regression network to obtain Big Five personality trait scores. Furthermore, we introduce a novel loss function to address the problem of training inaccuracies, taking advantage of its unique property of exhibiting a peak at the critical mean. Our tests on the ChaLearn First Impressions V2 multi‐modal dataset show partial performance surpassing state‐of‐the‐art networks.

Multi-characters Fusion Recognition System Based on Neural Network

Face Recognition: an Approach Based on Feature Fusion and Neural Network

Adaptive information fusion network for multi‐modal personality recognition

Facial Expression Recognition Based on CMAC Neural Network

Multi-sensor Data Fusion Based on Dynamic Fuzzy Neural Network

Chinese Character Recognition Method Based On Multi-Features And Parallel Neural Network Computation

Fusion of Visible and Infrared Imagery for Face Recognition

Multi-Modal Fusion Emotion Recognition Method of Speech Expression Based on Deep Learning

Multi-biometrics Fusion Based on Dynamic Weighting of Characteristic Signal Quality

RBF Neural Network Application to Face Recognition

Facial Expression Recognition System Based On Deep Residual Fusion Neural Network

Research of Printed Character Recognition Based on Neural Network Information Fusion

Multimodal emotion recognition based on deep neural network

Bimodal speaker identification using dynamic bayesian network

Research on facial expression recognition based on Multimodal data fusion and neural network

Multi-Net Fusion: Exploring a Brain-Inspired Neural Network Model for Facial Expression Recognition

Access Control by RFID and Face Recognition Based on Neural Network.

A Novel Approach for Character Feature Extraction and Recognition Based on RBF Neural Network

Facial Expression Recognition System Based on Multiple Feature Integration

Multi-Camera Human-Face Personal Identification System Based on the BIOMIMETIC PATTERN RECOGNITION

Multimodal Recognition Based on Fusion of Ear and Profile Face