Optimization of Recurrent Neural Networks on Natural Language Processing.

Jingyu Huang,Yunfei Feng
DOI: https://doi.org/10.1145/3373509.3373573
2019-01-01
Abstract:Recurrent neural networks (RNNs) are successful in various sequence modelling tasks, including natural language processing (NLP). However, parallelising the training process of RNNs proves to be difficult as computation at each timestep is dependent upon the output of the previous timestep. Many approaches have been taken to improve parallelisation of RNNs in recent years and achieved remarkable results. This paper investigate ways to further improve training speed and accuracy of RNNs in sentiment classification by combining methods of improving recurrent structure and recurrent units. Results show that not all combinations can result in an improvement, but rather significant improvements can be produced with the right arrangement of recurrent models.
What problem does this paper attempt to address?