Research on the Application of Deep Learning-based BERT Model in Sentiment Analysis

Yichao Wu,Zhengyu Jin,Chenxi Shi,Penghao Liang,Tong Zhan

2024-03-13

Abstract:This paper explores the application of deep learning techniques, particularly focusing on BERT models, in sentiment analysis. It begins by introducing the fundamental concept of sentiment analysis and how deep learning methods are utilized in this domain. Subsequently, it delves into the architecture and characteristics of BERT models. Through detailed explanation, it elucidates the application effects and optimization strategies of BERT models in sentiment analysis, supported by experimental validation. The experimental findings indicate that BERT models exhibit robust performance in sentiment analysis tasks, with notable enhancements post fine-tuning. Lastly, the paper concludes by summarizing the potential applications of BERT models in sentiment analysis and suggests directions for future research and practical implementations.

Machine Learning,Computation and Language

What problem does this paper attempt to address?

The paper primarily explores the application of the deep learning-based BERT model in sentiment analysis. Specifically, the research objectives include the following aspects:

1. **Exploring the effectiveness of the BERT model in sentiment analysis tasks**: Through in-depth analysis and experimental validation, the study aims to reveal the effectiveness and potential limitations of the BERT model in identifying different text sentiment tendencies.

2. **Comparing the performance of the BERT model with traditional word vector models**: The paper compares the BERT model (particularly the DistilBERT version) with traditional word vector models (such as FastText, Word2Vec, and GloVe) to evaluate their performance in sentiment analysis tasks.

3. **Methods to optimize the BERT model**: The research also explores methods to improve the BERT model, including enhancing the hidden language model, introducing denoising autoencoders, and incorporating alternative word detection techniques to further improve the model's performance in sentiment analysis tasks.

4. **Experimental validation**: In the experimental section, researchers used the SST2 dataset (Stanford Sentiment Treebank) to train and test the models, comparing the performance of the sentiment classification model using DistilBERT with other traditional models. The results show that even without the fine-tuning phase, DistilBERT still outperforms traditional models; and by fine-tuning the DistilBERT model, the scores for downstream tasks can be further improved.

In summary, this study aims to experimentally validate the potential of the BERT model and its variants in the field of sentiment analysis and proposes some optimization methods to enhance model performance.