Local Bidirectional Long Short Term Memory for Text Classification

Shengxian WAN,Yanyan LAN,Jiafeng GUO,Jun XU,Liang PANG,Xueqi CHENG
2017-01-01
Abstract:Deep learning has shown great benefits for natural language processing in recent years.Models such as Recurrent Neural Networks (RNNs) have been proposed to extract text representation,which can be applied for text classification.Long short term memory (LSTM) is an advanced kind of RNN with special neural cells.LSTM accepts a sequence of words from a sentence scans over the whole sequence and outputs the representation of the sentence.However,customary practices use only the last representation LSTM produced for classification,ignoring all other intermediate representations.A clear drawback is that it could not capture efficiently local features that are very important for determining the sentence's class label.In this paper,we propose the local bidirectional long short term memory to deal with this problem,including MaxBiLSTM and ConvBiLSTM.MaxBiLSTM conducts a max pooling operation and ConvBiLSTM conducts a convolution operation followed with a max pooling operation on all intermediate representations generated by bidirectional LSTM.Experimental results on two public datasets for text classification show that local bidirectional LSTM,especially ConvBiLSTM,outperforms bidirectional LSTM consistently and reaches the state-of-the-art performances.
What problem does this paper attempt to address?