Abstract:A challenging issue in the field of the automatic recognition of emotion from speech is the efficient modelling of long temporal contexts. Moreover, when incorporating long-term temporal dependencies between features, recurrent neural network (RNN) architectures are typically employed by default. In this work, we aim to present an efficient deep neural network architecture incorporating Connectionist Temporal Classification (CTC) loss for discrete speech emotion recognition (SER). Moreover, we also demonstrate the existence of further opportunities to improve SER performance by exploiting the properties of convolutional neural networks (CNNs) when modelling contextual information. Our proposed model uses parallel convolutional layers (PCN) integrated with Squeeze-and-Excitation Network (SEnet), a system herein denoted as PCNSE, to extract relationships from 3D spectrograms across timesteps and frequencies; here, we use the log-Mel spectrogram with deltas and delta-deltas as input. In addition, a self-attention Residual Dilated Network (SADRN) with CTC is employed as a classification block for SER. To the best of the authors' knowledge, this is the first time that such a hybrid architecture has been employed for discrete SER. We further demonstrate the effectiveness of our proposed approach on the Interactive Emotional Dyadic Motion Capture (IEMOCAP) and FAU-Aibo Emotion corpus (FAU-AEC). Our experimental results reveal that the proposed method is well-suited to the task of discrete SER, achieving a weighted accuracy (WA) of 73.1% and an unweighted accuracy (UA) of 66.3% on IEMOCAP, as well as a UA of 41.1% on the FAU-AEC dataset.

Representation Learning with Spectro-Temporal-Channel Attention for Speech Emotion Recognition

Exploring Spatio-Temporal Representations by Integrating Attention-based Bidirectional-LSTM-RNNs and FCNs for Speech Emotion Recognition

Attention-Enhanced Connectionist Temporal Classification for Discrete Speech Emotion Recognition

Deep Spectrum Feature Representations for Speech Emotion Recognition

DSTCNet: Deep Spectro-Temporal-Channel Attention Network for Speech Emotion Recognition.

Self-attention Transfer Networks for Speech Emotion Recognition

Time-Frequency Deep Representation Learning for Speech Emotion Recognition Integrating Self-attention.

Speech Emotion Recognition Based on Convolutional Neural Network with Attention-Based Bidirectional Long Short-Term Memory Network and Multi-Task Learning

Attention Based Fully Convolutional Network for Speech Emotion Recognition

Efficient Channel Attention for Speech Emotion Recognition with Asymmetric Convolution Module

Speech Emotion Recognition Via CNN-Transformer and Multidimensional Attention Mechanism

Temporal Attention Convolutional Network for Speech Emotion Recognition with Latent Representation.

Combining a parallel 2D CNN with a self-attention Dilated Residual Network for CTC-based discrete speech emotion recognition

An Attention Pooling Based Representation Learning Method for Speech Emotion Recognition

Speech Emotion Recognition using Channel Attention Mechanism

CM-TCN: Channel-Aware Multi-scale Temporal Convolutional Networks for Speech Emotion Recognition.

Convolutional Neural Network with Spectrogram and Perceptual Features for Speech Emotion Recognition

3-D Convolutional Recurrent Neural Networks with Attention Model for Speech Emotion Recognition.

Spatiotemporal and Frequential Cascaded Attention Networks for Speech Emotion Recognition.

Time-Frequency Representation Learning with Graph Convolutional Network for Dialogue-Level Speech Emotion Recognition

Learning Salient Features for Speech Emotion Recognition Using CNN