Residual Recurrent Neural Network with Sparse Training for Offline Arabic Handwriting Recognition

Ruijie Yan,Liangrui Peng,GuangXiang Bin,Shengjin Wang,Yao Cheng
DOI: https://doi.org/10.1109/icdar.2017.171
2017-01-01
Abstract:Deep Recurrent Neural Networks (RNN) have been suffering from the overfitting problem due to the model redundancy of the network structures. We propose a novel temporal and spatial residual learning method for RNN, followed with sparse training by weight pruning to gain sparsity in network parameters. For a Long Short-Term Memory (LSTM) network, we explore the combination schemes and parameter settings for temporal and spatial residual learning with sparse training. Experiments are carried out on the IFN/ENIT database. For the character error rate on the testing set e while training with sets a, b, c, d, the previously reported best result is 13.42%, and the proposed configuration of temporal residual learning followed with sparse training achieves the state-of-the-art result 12.06%.
What problem does this paper attempt to address?