Abstract:The Multimodal Sentiment Analysis Challenge (MuSe) 2022 is dedicated to multimodal sentiment and emotion recognition. For this year's challenge, we feature three datasets: (i) the Passau Spontaneous Football Coach Humor (Passau-SFCH) dataset that contains audio-visual recordings of German football coaches, labelled for the presence of humour; (ii) the Hume-Reaction dataset in which reactions of individuals to emotional stimuli have been annotated with respect to seven emotional expression intensities, and (iii) the Ulm-Trier Social Stress Test (Ulm-TSST) dataset comprising of audio-visual data labelled with continuous emotion values (arousal and valence) of people in stressful dispositions. Using the introduced datasets, MuSe 2022 2022 addresses three contemporary affective computing problems: in the Humor Detection Sub-Challenge (MuSe-Humor), spontaneous humour has to be recognised; in the Emotional Reactions Sub-Challenge (MuSe-Reaction), seven fine-grained `in-the-wild' emotions have to be predicted; and in the Emotional Stress Sub-Challenge (MuSe-Stress), a continuous prediction of stressed emotion values is featured. The challenge is designed to attract different research communities, encouraging a fusion of their disciplines. Mainly, MuSe 2022 targets the communities of audio-visual emotion recognition, health informatics, and symbolic sentiment analysis. This baseline paper describes the datasets as well as the feature sets extracted from them. A recurrent neural network with LSTM cells is used to set competitive baseline results on the test partitions for each sub-challenge. We report an Area Under the Curve (AUC) of .8480 for MuSe-Humor; .2801 mean (from 7-classes) Pearson's Correlations Coefficient for MuSe-Reaction, as well as .4931 Concordance Correlation Coefficient (CCC) and .4761 for valence and arousal in MuSe-Stress, respectively.

JTMA: Joint Multimodal Feature Fusion and Temporal Multi-head Attention for Humor Detection

AMTN: Attention-Enhanced Multimodal Temporal Network for Humor Detection

Multimodal Cross-Lingual Features and Weight Fusion for Cross-Cultural Humor Detection

Hybrid Multimodal Feature Extraction, Mining and Fusion for Sentiment Analysis

Modality-invariant Temporal Representation Learning for Multimodal Sentiment Classification

DPP: A Dual-Phase Processing Method for Cross-Cultural Humor Detection

Multimodal Temporal Attention in Sentiment Analysis

CEFM: CLIP Encoded Fusion Model for multimodal humor recognition on memes

Multimodal Emotion Recognition and Sentiment Analysis via Attention Enhanced Recurrent Model

Temporal-aware Multimodal Feature Fusion for Sentiment Analysis

The MuSe 2024 Multimodal Sentiment Analysis Challenge: Social Perception and Humor Recognition

Multimodal Sentiment Analysis Based on Recurrent Neural Network and Multimodal Attention.

Multi-modal Fusion for Video Sentiment Analysis

A Multimodal Sentiment Analysis Approach Based on a Joint Chained Interactive Attention Mechanism

ABML: Attention-Based Multi-Task Learning for Jointly Humor Recognition and Pun Detection

Multi-modal Sentiment and Emotion Joint Analysis with a Deep Attentive Multi-task Learning Model

Multimodal Sentiment Analysis Based on Cross-Modal Attention and Gated Cyclic Hierarchical Fusion Networks

The MuSe 2022 Multimodal Sentiment Analysis Challenge: Humor, Emotional Reactions, and Stress

Towards Multimodal Prediction of Spontaneous Humour: A Novel Dataset and First Results

Video Sentiment Analysis with Bimodal Information-augmented Multi-Head Attention