Abstract:There has been a significant focus on modelling emotion ambiguity in recent years, with advancements made in representing emotions as distributions to capture ambiguity. However, there has been comparatively less effort devoted to the consideration of temporal dependencies in emotion distributions which encodes ambiguity in perceived emotions that evolve smoothly over time. Recognizing the benefits of using constrained dynamical neural ordinary differential equations (CD-NODE) to model time series as dynamic processes, we propose an ambiguity-aware dual-constrained Neural ODE approach to model the dynamics of emotion distributions on arousal and valence. In our approach, we utilize ODEs parameterised by neural networks to estimate the distribution parameters, and we integrate additional constraints to restrict the range of the system outputs to ensure the validity of predicted distributions. We evaluated our proposed system on the publicly available RECOLA dataset and observed very promising performance across a range of evaluation metrics.

What problem does this paper attempt to address?

The paper attempts to address the issue of modeling emotion ambiguity and its temporal dependency in continuous emotion prediction. Specifically, the authors focus on the following points: 1. **Modeling Emotion Ambiguity**: In recent years, researchers have noticed the ambiguity in emotion labels (i.e., differences between different raters) and have tried to capture this ambiguity by representing emotions as distributions. However, most existing methods mainly focus on handling static emotion distributions, neglecting the dynamic nature of these distributions over time. 2. **Considering Temporal Dependency**: Emotion is a dynamic process that changes smoothly over time. Therefore, emotion distributions are not only dynamic themselves, but their ambiguity also changes over time. Many existing methods do not fully consider these temporal dependencies when modeling emotion distributions. 3. **Application of Constraints**: To ensure the validity and reasonableness of the predicted emotion distributions, certain constraints need to be applied to the model outputs. For example, smoothness constraints can ensure that changes in emotional states are smooth, and range constraints can ensure that the predicted distribution parameters are within a reasonable range. To address the above issues, the authors propose a new Constrained Dynamic Neural Ordinary Differential Equation (CD-NODE γ) system. This system uses neural network-parameterized ordinary differential equations (ODEs) to estimate the parameters of emotion distributions and introduces smoothness and range constraints to ensure that the predicted emotion distributions are both valid and reasonable. Experimental results show that this system performs well on multiple evaluation metrics, particularly in terms of prediction accuracy in low-ambiguity regions and reliability in high-ambiguity regions.

Dual-Constrained Dynamical Neural ODEs for Ambiguity-aware Continuous Emotion Prediction

Continuous Emotion Ambiguity Prediction: Modeling with Beta Distributions

Exploring Spatio-Temporal Representations by Integrating Attention-based Bidirectional-LSTM-RNNs and FCNs for Speech Emotion Recognition

Time-Delay Neural Network for Continuous Emotional Dimension Prediction from Facial Expression Sequences.

Bridging the Emotional Semantic Gap via Multimodal Relevance Estimation

Beyond Predictions in Neural ODEs: Identification and Interventions.

Handling Ambiguity in Emotion: From Out-of-Domain Detection to Distribution Estimation

Latent Space Energy-based Neural ODEs

A multimodal shared network with a cross-modal distribution constraint for continuous emotion recognition

Continuous Multimodal Emotion Prediction Based on Long Short Term Memory Recurrent Neural Network

Modeling emotion in complex stories: the Stanford Emotional Narratives Dataset

Tracking Emotions Using an Evolutionary Model of Mental State Transitions: Introducing a New Paradigm

Efficient Modeling of Long Temporal Contexts for Continuous Emotion Recognition.

Dynamic Graph Neural Ordinary Differential Equation Network for Multi-modal Emotion Recognition in Conversation

Bridging Discrete and Continuous: A Multimodal Strategy for Complex Emotion Detection

Deep Emotional Arousal Network for Multimodal Sentiment Analysis and Emotion Recognition

Learning Chaotic Systems and Long-Term Predictions with Neural Jump ODEs

CDL: Curriculum Dual Learning for Emotion-Controllable Response Generation

Dive into Ambiguity: Latent Distribution Mining and Pairwise Uncertainty Estimation for Facial Expression Recognition.

Neural Controlled Differential Equations for Online Prediction Tasks

Dynamic-Attention-based EEG State Transition Modeling for Emotion Recognition