Abstract:Speech emotion recognition is challenging because of the affective gap between the subjective emotions and low-level features. Integrating multilevel feature learning and model training, deep convolutional neural networks (DCNN) has exhibited remarkable success in bridging the semantic gap in visual tasks like image classification, object detection. This paper explores how to utilize a DCNN to bridge the affective gap in speech signals. To this end, we first extract three channels of log Mel-spectrograms (static, delta, and delta delta) similar to the red, green, blue (RGB) image representation as the DCNN input. Then, the AlexNet DCNN model pretrained on the large ImageNet dataset is employed to learn high-level feature representations on each segment divided from an utterance. The learned segment-level features are aggregated by a discriminant temporal pyramid matching (DTPM) strategy. DTPM combines temporal pyramid matching and optimal Lp-norm pooling to form a global utterance-level feature representation, followed by the linear support vector machines for emotion classification. Experimental results on four public datasets, that is, EMO-DB, RML, eNTERFACE05, and BAUM-1s, show the promising performance of our DCNN model and the DTPM strategy. Another interesting finding is that the DCNN model pretrained for image applications performs reasonably good in affective speech feature extraction. Further fine tuning on the target emotional speech datasets substantially promotes recognition performance.

A Novel Speech Emotion Recognition Method via Transfer PCA and Sparse Coding.

Speech Emotion Recognition Based On Sparse Transfer Learning Method

Exploring Spatio-Temporal Representations by Integrating Attention-based Bidirectional-LSTM-RNNs and FCNs for Speech Emotion Recognition

Transfer Semi-Supervised Non-Negative Matrix Factorization For Speech Emotion Recognition

Speech Emotion Recognition Using Transfer Non-Negative Matrix Factorization

Speech Emotion Recognition Based on Syllable-Level Feature Extraction

Cross-corpus speech emotion recognition using transfer semi-supervised discriminant analysis

Speech Emotion Recognition Based on Multi-task Deep Feature Extraction and MKPCA Feature Fusion

Cross-corpus Speech Emotion Recognition Based on Transfer Non-Negative Matrix Factorization

Learning Corpus-Invariant Discriminant Feature Representations for Speech Emotion Recognition.

Speech Emotion Recognition Based on Linear Discriminant Analysis and Support Vector Machine Decision Tree

Real-time Speech Emotion Recognition Based on Syllable-Level Feature Extraction

Speech emotion recognition using PCA-based spectral features and HMM

Transfer Sparse Discriminant Subspace Learning for Cross-Corpus Speech Emotion Recognition

Speech Emotion Recognition Based on Robust Discriminative Sparse Regression

Reduce the dimensions of emotional features by principal component analysis for speech emotion recognition

Speech Emotion Recognition Method Based on Hidden Factor Analysis

A Discriminative Feature Representation Method Based on Cascaded Attention Network With Adversarial Strategy for Speech Emotion Recognition

Learning Deep Multimodal Affective Features for Spontaneous Speech Emotion Recognition.

A New Learning Scheme of Emotion Recognition From Speech by Using Mean Fourier Parameters

Speech Emotion Recognition Using Deep Convolutional Neural Network and Discriminant Temporal Pyramid Matching