Pretrain Deep Models by Distant Supervision for Weibo Sentiment Analysis

Shengxian WAN,Yanyan LAN,Jiafeng GUO,Xueqi CHENG

2017-01-01

Abstract:Sentiment analysis (SA) is important in many applications such as commercial business and political election.The state-of-the-art methods of SA are based on shallow machine learning models.These methods are heavily dependent on feature engineering,however,the features for Weibo SA are difficult to be extracted manually.Deep learning (DL) can learn hierarchical representations from raw data automatically and has been applied for SA.Recently proposed DL techniques shown that one can train deep models successfully given enough supervised data.However,in Weibo SA,supervised data are usually too scarce.It is easy to obtain large scale distant supervision data in Weibo.In this paper,we proposed to pre-train deep models by distant supervision and used supervised data to fine-tune the deep models.This approach could take the advantages of distant supervision to learn good initial models while using supervised data to improve the models and to correct the errors brought by distant supervision.Experimental results on Sina Weibo dataset show that we can train deep models with small scale supervised data and obtain better results than shallow models.