Abstract:Speech emotion recognition (SER) has become a crucial topic in the field of human–computer interactions. Feature representation plays an important role in SER, but there are still many challenges in feature representation such as the inability to predict which features are most effective for SER and the cultural differences in emotion expression. Most previous studies use a single type of feature for the recognition task or conduct early fusion of features. However, a single type of feature cannot well reflect the emotions of speech signals. Also, different features contain different information, direct fusion cannot integrate the advantages of different features. To overcome these challenges, this paper proposes a parallel network for multi-scale SER based on a connection attention mechanism (AMSNet). AMSNet fuses fine-grained frame-level manual features with coarse-grained utterance-level deep features. Meanwhile, it adopts different speech emotion feature extraction modules according to the temporal and spatial features of speech signals, which enriches features and improves feature characterization. The network consists of a frame-level representation learning module (FRLM) based on the time structure and an utterance-level representation learning module (URLM) based on the global structure. Besides, improved attention-based long short-term memory (LSTM) is introduced into FRLM to focus on the frames that contribute more to the final emotion recognition result. In URLM, a convolutional neural network with the squeeze-and-excitation block (SCNN) is introduced to extract deep features. In addition, the connection attention mechanism is proposed for feature fusion, which applies different weights to different features. Extensive experiments are conducted on the IEMOCAP and EmoDB datasets, and the results demonstrate the effectiveness and performance superiority of AMSNet. Our code will be publicly available at https://codeocean.com/capsule/8636967/tree/v1.

Speech Topic Classification Based on Multi-Scale and Graph Attention Networks

Speech Topic Classification Based on Pre-trained and Graph Networks.

End-to-end Speech Topic Classification Based on Pre-Trained Model Wavlm

Deep Dual-Stream Network with Scale Context Selection Attention Module for Semantic Segmentation

Densely Connected CNN with Multi-scale Feature Attention for Text Classification

Topic Classification on Spoken Documents Using Deep Acoustic and Linguistic Features

An Attentional Spatial Temporal Graph Convolutional Network with Co-Occurrence Feature Learning for Action Recognition

Hierarchical Multi-Granularity Attention- Based Hybrid Neural Network for Text Classification.

Multi-layer Attention Mechanism for Speech Keyword Recognition

Multi-scale Feature Based Convolutional Neural Networks for Large Vocabulary Speech Recognition

A stacked convolutional neural network framework with multi-scale attention mechanism for text-independent voiceprint recognition

Monaural Multi-Talker Speech Recognition With Attention Mechanism And Gated Convolutional Networks

Multi-Scale Spatial-Temporal Attention Networks for Functional Connectome Classification

Stacked Multiscale Densely Connected Temporal Convolutional Attention Network for Multi-Objective Speech Enhancement in an Airborne Environment

Emotion Classification from Multi-Channel EEG Signals Using HiSTN: A Hierarchical Graph-based Spatial-Temporal Approach

HAIN: Multi-label Classification with Hierarchical Attention-based Interaction Network for Multi-turn Dialogue Texts

Inferring Speaking Styles from Multi-modal Conversational Context by Multi-scale Relational Graph Convolutional Networks

Multi-Scale Self-Attention for Text Classification

Stacked Residual Recurrent Neural Networks with Cross-Layer Attention for Text Classification.

Multi-label text classification based on semantic-sensitive graph convolutional network

Learning multi-scale features for speech emotion recognition with connection attention mechanism