Abstract:Sentiment analysis uses a series of automated cognitive methods to determine the author's or speaker's attitudes toward an expressed object or text's overall emotional tendencies. In recent years, the growing scale of opinionated text from social networks has brought significant challenges to humans' sentimental tendency mining. The pretrained language model designed to learn contextual representation achieves better performance than traditional learning word vectors. However, the existing two basic approaches for applying pretrained language models to downstream tasks, feature-based and fine-tuning methods, are usually considered separately. What is more, different sentiment analysis tasks cannot be handled by the single task-specific contextual representation. In light of these pros and cons, we strive to propose a broad multitask transformer network (BMT-Net) to address these problems. BMT-Net takes advantage of both feature-based and fine-tuning methods. It was designed to explore the high-level information of robust and contextual representation. Primarily, our proposed structure can make the learned representations universal across tasks via multitask transformers. In addition, BMT-Net can roundly learn the robust contextual representation utilized by the broad learning system due to its powerful capacity to search for suitable features in deep and broad ways. The experiments were conducted on two popular datasets of binary Stanford Sentiment Treebank (SST-2) and SemEval Sentiment Analysis in Twitter (Twitter). Compared with other state-of-the-art methods, the improved representation with both deep and broad ways is shown to achieve a better F1-score of 0.778 in Twitter and accuracy of 94.0% in the SST-2 dataset, respectively. These experimental results demonstrate the abilities of recognition in sentiment analysis and highlight the significance of previously overlooked design decisions about searching contextual features in deep and broad spaces.

Transformer and Multi-scale Convolution for Target-Oriented Sentiment Analysis

TensorFormer: A Tensor-Based Multimodal Transformer for Multimodal Sentiment Analysis and Depression Detection

SemiCVT: Semi-Supervised Convolutional Vision Transformer for Semantic Segmentation

Multi-depth Fusion Transformer and Batch Piecewise Loss for Visual Sentiment Analysis.

Transformer-Based Interactive Multi-Modal Attention Network for Video Sentiment Detection

A Hybrid Transformer and Attention Based Recurrent Neural Network for Robust and Interpretable Sentiment Analysis of Tweets

BMT-Net: Broad Multitask Transformer Network for Sentiment Analysis

Multi-scale Cooperative Multimodal Transformers for Multimodal Sentiment Analysis in Videos

Transformation Networks for Target-Oriented Sentiment Classification

Text Sentiment Analysis Based on Transformer and Augmentation

SS-Trans (Single-Stream Transformer for Multimodal Sentiment Analysis and Emotion Recognition): The Emotion Whisperer—A Single-Stream Transformer for Multimodal Sentiment Analysis

Sentiment-Aware Transformer Using Joint Training

Multi-Scale Temporal Transformer For Speech Emotion Recognition

Image and Text Aspect Level Multimodal Sentiment Classification Model Using Transformer and Multilayer Attention Interaction

Transformer-Based Feature Fusion Approach for Multimodal Visual Sentiment Recognition Using Tweets in the Wild

A Two-Stage Stacked Transformer Framework for Multimodal Sentiment Analysis

Multilayer interactive attention bottleneck transformer for aspect-based multimodal sentiment analysis

Transformer-based approaches to Sentiment Detection

Multi-Task Learning Model Based on Multi-Scale CNN and LSTM for Sentiment Classification

TEDT: Transformer-Based Encoding–Decoding Translation Network for Multimodal Sentiment Analysis

A transformer-encoder-based multimodal multi-attention fusion network for sentiment analysis