Abstract:Sentiment plays an important role in cognition, creativity, attention, and decision making in people's daily lives. Researchers have made great progress in sentiment recognition through images and speech. In this paper, a multimodal dataset is proposed for sentiment classification (MDSTC1)-a multimodal dataset collected with multimodal channels by customized physiological sensors and manually labelled for human sentiment analysis. MDSTC contains galvanic skin response (GSR), pulse, speech, and facial expression data of 100 volunteers labelled with timeline, emotional self-ratings, and personal information such as age, sex and Big Five personality scores, which were collected while watching video clips to stimulate emotions. Using the MDSTC dataset, the GSR channel with six types of emotion labels is used to perform human sentiment analysis in this paper. The GSR signal is converted into a spectrogram to adopt image-related methods and deep learning models. A convolutional neural network, long short-term memory, and self-attention mechanism are adopted and combined, and spatiotemporal hybrid models are proposed to perform human sentiment analysis. Comparisons are performed between the proposed model and state-of-the-art models in related works. The experimental results show that with the proposed spatiotemporal hybrid models, better results are obtained with respect to precision, recall, and F1-score. The experimental results also show that with the proposed spatiotemporal hybrid models working with GSR, people's emotional changes can be obtained in real time with high precision. (C) 2019 Elsevier B.V. All rights reserved.

Sentiment Classification And Personality Detection Via Galvanic Skin Response Based on Deep Learning Models

Hybrid Spatiotemporal Models for Sentiment Classification Via Galvanic Skin Response

Dynamic Facial Features in Positive-Emotional Speech for Identification of Depressive Tendencies

A sentiment-aware deep learning approach for personality detection from text

Emotion Recognition from Galvanic Skin Response Signal Based on Deep Hybrid Neural Networks

Deep learning-based personality recognition from text posts of online social networks

Sentimental Prediction Model of Personality Based on CNN-LSTM in a Social Media Environment

PS-GCN: psycholinguistic graph and sentiment semantic fused graph convolutional networks for personality detection

Personality First in Emotion: a Deep Neural Network Based on Electroencephalogram Channel Attention for Cross-Subject Emotion Recognition.

Integrating audio and visual modalities for multimodal personality trait recognition via hybrid deep learning

Multimodal Video-based Apparent Personality Recognition Using Long Short-Term Memory and Convolutional Neural Networks

Using Machine Learning Based Models for Personality Recognition

PersonalitySensing: A Multi-View Multi-Task Learning Approach for Personality Detection Based on Smartphone Usage

Personality Trait Recognition using ECG Spectrograms and Deep Learning

Multitask Learning for Emotion and Personality Detection

Personality Recognition Based On User Generated Content

User personality prediction based on topic preference and sentiment analysis using LSTM model

Who Am I? Personality Detection Based On Deep Learning For Texts

2.5D Facial Personality Prediction Based on Deep Learning

Your Face Mirrors Your Deepest Beliefs-Predicting Personality and Morals through Facial Emotion Recognition

Personality Classification of Social Users Based on Feature Fusion