Sentiment Analysis Using Autoregressive Language Modeling and Broad Learning System

Xin-rong Gong,Jian-xiu Jin,Tong Zhang
DOI: https://doi.org/10.1109/bibm47256.2019.8983025
2019-01-01
Abstract:In the era of big data, mining the emotional tendency of opinions through natural language processing technology is meaningful for the timely understanding of human behavior data. Nowadays, Generalized Autoregressive Pre-training Language Modeling (XLNet) can not only capture bidirectional contextual knowledge but also learn the word dependency. However, its sentence-level representation didn't take broad features into account. In this paper, we design a novel architecture, called Broad Autoregressive Language Model (BroXLNet), to automatically process sentiment analysis task. BroXLNet integrates the advantage of generalized autoregressive language modeling and broad learning system, which has the ability of extracting deep contextual features and randomly searching high-level contextual representation in broad spaces. We evaluated our algorithm on binary Stanford Sentiment Treebank dataset. Compared with the state-of-the-art methods, e.g., BiLSTM, ELMo, OpenAI GPT, BERT and XLNet, BroXLNet achieved the best result of 94.0% in sentiment analysis task of binary Stanford Sentiment Treebank. The result demonstrated the excellent classifying ability of BroXLNet in sentiment analysis.
What problem does this paper attempt to address?