Abstract:As an important part of emotion research, facial expression recognition is a necessary condition for intelligent interaction between human and machine, which has an important research significance and a potential commercial value. Convolutional neural network (CNN) is an effective method to recognize facial emotions, which can perform feature extraction and classification simultaneously, and can automatically discover multiple levels of representations in data. Due to the fact that there are millions of parameters involved in training the convolutional neural network model, and there is a large demand for marked samples, transfer learning is often used to fine-tune the pre-trained model for a small target sample set. However, there are often some content differences between data sets during deep transfer learning, which will affect the recognition ability of feature extraction. In order to improve the facial expression recognition ability in transfer learning, a hybrid transfer learning model based on an improved convolution restricted boltzmann machine (CRBM) model and a CNN model is proposed in this paper. This method is fused by two learning abilities of these two models. When the pre-trained CNN model is transferred to a small target set, the CRBM is used to replace the full connection layer in the CNN model, and the CRBM layer and the sofmax layer will be retrained on the target set. The added CRBM layer can not only fully connects all feature maps, but can also learn about the unique statistical characteristics about the target set, which eliminates the influence of content differences between data sets, and extracts higher-order statistical features of facial expression images from the target set. The proposed method is evaluated based on four publicly available facial expression databases: JAFFE, FER2013, SFEW and RAF-DB. The new method can achieve better performance than most state-of-the-art methods, and it can effectively prevent the negative influence of transfer learning features between different data sets.

Correction To: Deep Transfer Learning for Image Emotion Analysis: Reducing Marginal and Joint Distribution Discrepancies Together

Joint Image Emotion Classification and Distribution Learning Via Deep Convolutional Neural Network.

Exploring Discriminative Representations for Image Emotion Recognition with CNNs.

Discussions of Different Deep Transfer Learning Models for Emotion Recognitions

The Application of a Hybrid Transfer Algorithm Based on a Convolutional Neural Network Model and an Improved Convolution Restricted Boltzmann Machine Model in Facial Expression Recognition

A Saliency Detection and Gram Matrix Transform-Based Convolutional Neural Network for Image Emotion Classification.

Image Emotion Distribution Learning With Graph Convolutional Networks

Deep Emotion Transfer Network for Cross-database Facial Expression Recognition

Learning Multi-level Representations for Image Emotion Recognition in the Deep Convolutional Network

The application of deep learning in computer vision: Facial emotion recognition based on convolutional neural network

Feature Discrimination and Diversification for Image Emotion Recognition

A critical examination of deficiencies in the convolutional neural network model for facial emotion recognition

EmotionGAN: Unsupervised Domain Adaptation for Learning Discrete Probability Distributions of Image Emotions.

Dependency Exploitation: A Unified CNN-RNN Approach for Visual Emotion Recognition

A Transfer Learning Based Boosting Model for Emotion Analysis

Speech Emotion Recognition Using Deep Convolutional Neural Network and Discriminant Temporal Pyramid Matching

Improving the Performance of Sentiment Classification on Imbalanced Datasets with Transfer Learning.

CNN-Based Broad Learning for Cross-Domain Emotion Classification

Deep Learning Based Transfer Learning for Possible Facial Psychological Expression Recognition

Adaptively Learning Facial Expression Representation via C-F Labels and Distillation

Weakly Supervised Learning of Image Emotion Analysis Based on Cross-spatial Pooling.