Abstract:Multi-view speech emotion recognition (SER) based on the pre-trained model has gained attention in the last two years, which shows great potential in improving the model performance in speaker-independent scenarios. However, the existing work either relies on various fine-tuning methods or uses excessive feature views with complex fusion strategies, causing the increase of complexity with limited performance benefit. In this paper, we improve multi-view SER based on the pre-trained model from the perspective of a low-level speech feature. Specifically, we forgo fine-tuning the pre-trained model and instead focus on learning effective features hidden in the low-level speech feature mel-scale frequency cepstral coefficient (MFCC). We propose a t wo- s tream p ooling c hannel a ttention ( TsPCA ) module to discriminatively weight the channel dimensions of the features derived from MFCC. This module enables inter-channel interaction and learning of emotion sequence information across channels. Furthermore, we design a simple but effective feature view fusion strategy to learn robust representations. In the comparison experiments, our method achieves the WA and UA of 73.97%/74.69% and 74.61%/75.66% on the IEMOCAP dataset, 97.21% and 97.11% on the Emo-DB dataset, 77.08% and 77.34% on the RAVDESS dataset, and 74.38% and 71.43% on the SAVEE dataset. Extensive experiments on the four datasets demonstrate that our method consistently surpasses existing methods and achieves a new state-of-the-art result.

Syllable Level Speech Emotion Recognition Based on Formant Attention

Speech Emotion Recognition Based on Syllable-Level Feature Extraction

Real-time Speech Emotion Recognition Based on Syllable-Level Feature Extraction

Attention-Enhanced Connectionist Temporal Classification for Discrete Speech Emotion Recognition

Speech Emotion Recognition Based on Formant Characteristics Feature Extraction and Phoneme Type Convergence.

Speech Emotion Recognition Based on Convolutional Neural Network with Attention-Based Bidirectional Long Short-Term Memory Network and Multi-Task Learning

Self-attention Transfer Networks for Speech Emotion Recognition

Speaker-Independent Speech Emotion Recognition Based On Cnn-Blstm And Multiple Svms

Speech Emotion Recognition by Combining a Unified First-Order Attention Network with Data Balance

Speech Emotion Recognition Exploiting ASR-based and Phonological Knowledge Representations.

Speech Emotion Recognition Based on Clustering Assistance

Speech Emotion Recognition Based on Acoustic Segment Model.

Cross-Corpus Speech Emotion Recognition Based on Hybrid Neural Networks

A Time-Frequency Attention Mechanism with Subsidiary Information for Effective Speech Emotion Recognition

Pre-trained Deep Convolution Neural Network Model With Attention for Speech Emotion Recognition

Improving Pre-trained Model-based Speech Emotion Recognition from a Low-level Speech Feature Perspective

Multimodal Emotion Recognition from Raw Audio with Sinc-convolution

Speech emotion recognition based on emotion perception

SELM: Enhancing Speech Emotion Recognition for Out-of-Domain Scenarios

Speech Emotion Recognition with Multiscale Area Attention and Data Augmentation