Abstract:Sentiment analysis (SA) detects people’s opinions from text engaging natural language processing (NLP) techniques. Recent research has shown that deep learning models, i.e., Convolutional Neural Network (CNN), Recurrent Neural Network (RNN), and Transformer-based provide promising results for recognizing sentiment. Nonetheless, CNN has the advantage of extracting high-level features by using convolutional and max-pooling layers; it cannot efficiently learn a sequence of correlations. At the same time, Bidirectional RNN uses two RNN directions to improve extracting long-term dependencies. However, it cannot extract local features in parallel, and Transformer-based like Bidirectional Encoder Representations from Transformers (BERT) are the computational resources needed to fine-tune, facing an overfitting problem on small datasets. This paper proposes a novel attention-based model that utilizes CNNs with LSTM (named ACL-SA). First, it applies a preprocessor to enhance the data quality and employ term frequency-inverse document frequency (TF-IDF) feature weighting and pre-trained Glove word embedding approaches to extract meaningful information from textual data. In addition, it utilizes CNN’s max-pooling to extract contextual features and reduce feature dimensionality. Moreover, it uses an integrated bidirectional LSTM to capture long-term dependencies. Furthermore, it applies the attention mechanism at the CNN’s output layer to emphasize each word’s attention level. To avoid overfitting, the Guasiannoise and GuasianDroupout are adopted as regularization. The model’s robustness is evaluated on four English standard datasets, i.e., Sentiment140, US-airline, Sentiment140-MV, SA4A with various performance matrices, and compared efficiency with existing baseline models and approaches. The experiment results show that the proposed method significantly outperforms the state-of-the-art models.

S2SAN: A sentence-to-sentence attention network for sentiment analysis of online reviews

SAAN: A Sentiment-Aware Attention Network for Sentiment Analysis.

A Relative Position Attention Network for Aspect-Based Sentiment Analysis

Self-Attention Networks and Adaptive Support Vector Machine for aspect-level sentiment classification

DiSAN: Directional Self-Attention Network for RNN/CNN-Free Language Understanding

LAAN: A Linguistic-Aware Attention Network for Sentiment Analysis

High accuracy offering attention mechanisms based deep learning approach using CNN/bi-LSTM for sentiment analysis

Multiple Structural Priors Guided Self Attention Network for Language Understanding

Attention-Based CNN and Bi-LSTM Model Based on TF-IDF and GloVe Word Embedding for Sentiment Analysis

Reinforced Self-Attention Network: a Hybrid of Hard and Soft Attention for Sequence Modeling

Sentiment analysis with adaptive multi-head attention in Transformer

A Lexicon-Based Supervised Attention Model for Neural Sentiment Analysis.

Multi-Task Multi-Head Attention Memory Network for Fine-Grained Sentiment Analysis.

Sentence Constituent-Aware Aspect-Category Sentiment Analysis with Graph Attention Networks

SG-Net: Syntax-Guided Machine Reading Comprehension.

Hierarchical Neural Network with Serial Attention Mechanism for Review Sentiment Classifification

A Window-Based Self-Attention Approach for Sentence Encoding

Sentiment Lexicon Enhanced Neural Sentiment Classification

Text Sentiment Analysis Based on a New Hybrid Network Model

Amplifying Aspect-Sentence Awareness: A Novel Approach for Aspect-Based Sentiment Analysis