Multi-Column Deep Neural Networks for Offline Handwritten Chinese Character Classification

Dan Cireşan,Jürgen Schmidhuber

DOI: https://doi.org/10.48550/arXiv.1309.0261

2013-09-02

Abstract:Our Multi-Column Deep Neural Networks achieve best known recognition rates on Chinese characters from the ICDAR 2011 and 2013 offline handwriting competitions, approaching human performance.

Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

SmartHandwriting: Handwritten Chinese Character Recognition with Smartwatch.

Jian Zhang,Hongliang Bi,Yanjiao Chen,Mingyu Wang,Liming Han,Ligan Cai

DOI: https://doi.org/10.1109/jiot.2019.2947448

IF: 10.6

2019-01-01

IEEE Internet of Things Journal

Abstract:Most existing systems use portable devices or image processing techniques for handwritten Chinese character recognition (HCCR), which are unable to detect character when writing on a paper or sensitive to lighting conditions. In this article, we present the design, implementation, and evaluation of a smartwatch-based HCCR system, called SmartHandwriting. To segment each Chinese character, we further analyze the hand movement between the handwriting gesture and the wrist movement gesture and propose a novel algorithm to distinguish the two types of gestures. Due to too many Chinese characters for classification, we utilize the data augmentation method for avoiding overfitting. Then, we build the HCCR model using the deep convolutional neural network (DCNN) method. The recognition accuracy of the Chinese characters is 96.0%, and extensive experiments confirm its effectiveness and robustness. Moreover, we also explore adverse factors that affect the recognition performance, which can be avoided in the future.
A New Hybrid-Parameter Recurrent Neural Network for Online Handwritten Chinese Character Recognition

Haiqing Ren,Weiqiang Wang,Xiwen Qu,Yuanqiang Cai

DOI: https://doi.org/10.1016/j.patrec.2019.10.001

IF: 4.757

2019-01-01

Pattern Recognition Letters

Abstract:The recurrent neural network (RNN) is appropriate for dealing with temporal sequences. In this paper, we present a deep RNN with new features and apply it for online handwritten Chinese character recognition. Compared with the existing RNN models, three innovations are involved in the proposed system. First, a new hidden layer function for RNN is proposed for learning temporal information better. we call it Memory Pool Unit (MPU). The proposed MPU has a simple architecture. Second, a new RNN architecture with hybrid parameter is presented, in order to increasing the expression capacity of RNN. The proposed hybrid-parameter RNN has parameter changes when calculating the iteration at temporal dimension. Third, we make a adaptation that all the outputs of each layer are stacked as the output of network. Stacked hidden layer states combine all the hidden layer states for increasing the expression capacity. Experiments are carried out on the IAHCC-UCAS2016 dataset and the CASIA-OLHWDB1.1 dataset. The experimental results show that the hybrid-parameter RNN obtain a better recognition performance with higher efficiency (fewer parameters and faster speed). And the proposed Memory Pool Unit is proved to be a simple hidden layer function and obtains a competitive recognition results.
Deep LSTM Networks for Online Chinese Handwriting Recognition

Li Sun,Tonghua Su,Ce Liu,Ruigang Wang

DOI: https://doi.org/10.1109/icfhr.2016.0059

2016-01-01

Abstract:Currently two heavy burdens are borne in online Chinese handwriting recognition: a large-scale training data needs to be annotated with the boundaries of each character and effective features should be handcrafted by domain experts. To relieve such issues, the paper presents a novel end-to-end recognition method based on recurrent neural networks. A mixture architecture of deep bidirectional Long Short-Term Memory (LSTM) layers and feed forward subsampling layers is used to encode the long contextual history trajectories. The Connectionist Temporal Classification (CTC) objective function makes it possible to train the model without providing alignment information between input trajectories and output strings. During decoding, a modified CTC beam search algorithm is devised to integrate the linguistic constraints wisely. Our method is evaluated both on test set and competition set of CASIA-OLHWDB 2. x. Comparing with state-of-the-art methods, over 30% relative error reductions are observed on test set in terms of both correct rate and accurate rate. Even to the more challenging competition set, better results can be achieved by our method if the out-of-vocabulary problem can be ignored.
Deep Convolutional Neural Network Based Hidden Markov Model for Offline Handwritten Chinese Text Recognition

Zi-Rui Wang,Jun Du,Jin-Shui Hu,Yu-Long Hu

DOI: https://doi.org/10.1109/acpr.2017.65

2017-01-01

Abstract:This paper proposes a novel segmentation-free approach using deep neural network based hidden Markov model (DNN-HMM) for offline handwritten Chinese text recognition. In the general Bayesian framework, three key issues are comprehensively investigated, namely feature extraction, character modeling, and language modeling. First, as for the feature extraction on the basis of each frame or sliding window, the gradient-based features are extracted for the DNN-based classifier. Second, the text line is sequentially modeled by HMMs with each representing one character class. Meanwhile the DNNbased classifier is adopted to calculate the posterior probability of all HMM states. Finally, the character n-gram language model is integrated with the DNN-HMM character model for the Bayesian decision. The experiments on the ICDAR 2013 competition task of CASIA-HWDB database show that the proposed approach can achieve the best published recognition results to our knowledge, yielding a character error rate (CER) of 6.50%, which significantly outperforms the previously best reported oversegmentation approach (with a CER of 9.25%) and the segmentation- free approach using multidimensional longshort term memory recurrent neural network (MDLSTM-RNN) approach (with a CER of 10.6%).
A Comprehensive Study of Hybrid Neural Network Hidden Markov Model for Offline Handwritten Chinese Text Recognition.

Zi-Rui Wang,Jun Du,Wen-Chao Wang,Jian-Fang Zhai,Jin-Shui Hu

DOI: https://doi.org/10.1007/s10032-018-0307-0

2018-01-01

International Journal on Document Analysis and Recognition

Abstract:This paper proposes an effective segmentation-free approach using a hybrid neural network hidden Markov model (NN-HMM) for offline handwritten Chinese text recognition (HCTR). In the general Bayesian framework, the handwritten Chinese text line is sequentially modeled by HMMs with each representing one character class, while the NN-based classifier is adopted to calculate the posterior probability of all HMM states. The key issues in feature extraction, character modeling, and language modeling are comprehensively investigated to show the effectiveness of NN-HMM framework for offline HCTR. First, a conventional deep neural network (DNN) architecture is studied with a well-designed feature extractor. As for the training procedure, the label refinement using forced alignment and the sequence training can yield significant gains on top of the frame-level cross-entropy criterion. Second, a deep convolutional neural network (DCNN) with automatically learned discriminative features demonstrates its superiority to DNN in the HMM framework. Moreover, to solve the challenging problem of distinguishing quite confusing classes due to the large vocabulary of Chinese characters, NN-based classifier should output 19900 HMM states as the classification units via a high-resolution modeling within each character. On the ICDAR 2013 competition task of CASIA-HWDB database, DNN-HMM yields a promising character error rate (CER) of 5.24% by making a good trade-off between the computational complexity and recognition accuracy. To the best of our knowledge, DCNN-HMM can achieve a best published CER of 3.53%.
Multi-column Deep Neural Networks for Image Classification

Dan Cireşan,Ueli Meier,Juergen Schmidhuber

DOI: https://doi.org/10.48550/arXiv.1202.2745

2012-02-13

Abstract:Traditional methods of computer vision and machine learning cannot match human performance on tasks such as the recognition of handwritten digits or traffic signs. Our biologically plausible deep artificial neural network architectures can. Small (often minimal) receptive fields of convolutional winner-take-all neurons yield large network depth, resulting in roughly as many sparsely connected neural layers as found in mammals between retina and visual cortex. Only winner neurons are trained. Several deep neural columns become experts on inputs preprocessed in different ways; their predictions are averaged. Graphics cards allow for fast training. On the very competitive MNIST handwriting benchmark, our method is the first to achieve near-human performance. On a traffic sign recognition benchmark it outperforms humans by a factor of two. We also improve the state-of-the-art on a plethora of common image classification benchmarks.

Computer Vision and Pattern Recognition,Artificial Intelligence
Deep networks for online handwriting Chinese character recognition

Zhou Shusen,Chen Qingcai,Wang Xiaolong

2015-01-01

Abstract:In this paper, we propose a handwriting recognition with deep networks (HRDN) system for online handwriting Chinese character recognition based on deep learning method. The features of handwriting characters are abstracted by deep belief networks and coarse classified by an exponential loss function, then the characters are fine classified by modified quadratic discriminant function classifier. Experiments conducted on several standard online Chinese handwriting databases show that our proposed deep learning based method outperforms the state-of-the-art classifier. © 2015 ICIC International.
Offline Handwritten Chinese Text Recognition with Convolutional Neural Networks

Brian Liu,Xianchao Xu,Yu Zhang

DOI: https://doi.org/10.48550/arXiv.2006.15619

2020-06-28

Computer Vision and Pattern Recognition

Abstract:Deep learning based methods have been dominating the text recognition tasks in different and multilingual scenarios. The offline handwritten Chinese text recognition (HCTR) is one of the most challenging tasks because it involves thousands of characters, variant writing styles and complex data collection process. Recently, the recurrent-free architectures for text recognition appears to be competitive as its highly parallelism and comparable results. In this paper, we build the models using only the convolutional neural networks and use CTC as the loss function. To reduce the overfitting, we apply dropout after each max-pooling layer and with extreme high rate on the last one before the linear layer. The CASIA-HWDB database is selected to tune and evaluate the proposed models. With the existing text samples as templates, we randomly choose isolated character samples to synthesis more text samples for training. We finally achieve 6.81% character error rate (CER) on the ICDAR 2013 competition set, which is the best published result without language model correction.
A Study of Designing Compact Classifiers Using Deep Neural Networks for Online Handwritten Chinese Character Recognition

Jun Du,Jin-Shui Hu,Bo Zhu,Si Wei,Li-Rong Dai

DOI: https://doi.org/10.1109/icpr.2014.508

2014-01-01

Abstract:This paper presents a study of designing compact classifiers using deep neural networks for recognition of online handwritten Chinese characters. Two schemes are investigated based on practical considerations. First, deep neural networks are adopted purely as a classifier with a state-of-the-art feature extractor of online handwritten Chinese characters. Second, the so-called bottleneck features extracted from a bottleneck layer of deep neural networks are fed to the prototype-based classifier. The experiments on an in-house developed online Chinese handwriting corpus with a vocabulary of 15,167 characters show that compared with prototype-based classifier widely developed on the mobile device, deep neural network based classifier can yield significant improvements of recognition accuracy with acceptably increased footprint and latency while the bottleneck-feature approach can bring a more compact classifier with an observable performance gain.
Deep Matching Network for Handwritten Chinese Character Recognition

Zhiyuan Li,Qi Wu,Yi Xiao,Min Jin,Huaxiang Lu

DOI: https://doi.org/10.1016/j.patcog.2020.107471

IF: 8

2020-11-01

Pattern Recognition

Abstract:<p>Just like its remarkable achievements in many computer vision tasks, the convolutional neural networks (CNN) provide an end-to-end solution in handwritten Chinese character recognition (HCCR) with great success. However, the process of learning discriminative features for image recognition is difficult in cases where little data is available. In this paper, we propose a matching network which builds a connection between template characters and handwritten characters inspired by the human learning process of writing Chinese characters. The matching network replaces the parameters in the softmax regression layer with the features extracted from the template character images. After the training process has been finished, the powerful discriminative features help us to generalize the predictive power not just to new data, but to entire new Chinese characters that never appear in the training set before. Experiments performed on the ICDAR-2013 offline HCCR datasets have shown that the proposed method achieves a comparable performance to current CNN-based classifiers. Besides, the matching network has a very promising generalization ability to new Chinese characters that never appear in the existing training set.</p>

computer science, artificial intelligence,engineering, electrical & electronic
Improved Deep Convolutional Neural Network For Online Handwritten Chinese Character Recognition using Domain-Specific Knowledge

Weixin Yang,Lianwen Jin,Zecheng Xie,Ziyong Feng

DOI: https://doi.org/10.48550/arXiv.1505.07675

2015-05-28

Abstract:Deep convolutional neural networks (DCNNs) have achieved great success in various computer vision and pattern recognition applications, including those for handwritten Chinese character recognition (HCCR). However, most current DCNN-based HCCR approaches treat the handwritten sample simply as an image bitmap, ignoring some vital domain-specific information that may be useful but that cannot be learnt by traditional networks. In this paper, we propose an enhancement of the DCNN approach to online HCCR by incorporating a variety of domain-specific knowledge, including deformation, non-linear normalization, imaginary strokes, path signature, and 8-directional features. Our contribution is twofold. First, these domain-specific technologies are investigated and integrated with a DCNN to form a composite network to achieve improved performance. Second, the resulting DCNNs with diversity in their domain knowledge are combined using a hybrid serial-parallel (HSP) strategy. Consequently, we achieve a promising accuracy of 97.20% and 96.87% on CASIA-OLHWDB1.0 and CASIA-OLHWDB1.1, respectively, outperforming the best results previously reported in the literature.

Computer Vision and Pattern Recognition
A Residual-Attention Offline Handwritten Chinese Text Recognition Based on Fully Convolutional Neural Networks.

Yintong Wang,Yingjie Yang,Weiping Ding,Shuo Li

DOI: https://doi.org/10.1109/access.2021.3115606

IF: 3.9

2021-01-01

IEEE Access

Abstract:Offline handwritten Chinese text recognition is one of the most challenging tasks in that it involves various writing styles, complex character-touching, and large number of character categories. In this paper, we propose a residual-attention offline handwritten Chinese text recognition based on fully convolutional neural networks, which is segmentation-free handwritten recognition that avoids the impact of incorrect character segmentation. By designing a smart residual attention gate block, our model can help to extract important features, and effectively implement the training of deep convolutional neural networks. Furthermore, we deploy an expansion factor to indicate the trade-off between computing resources for model training and the ability of a gradient to propagate across multiple layers, and make our model training adapt to different computing platforms. Experiments on the CASIA-HWDB and ICDAR-2013 competition dataset show that our method achieves a competitive performance on offline handwritten Chinese text recognition. On the CASIA-HWDB test set, the character-level accurate rate and correct rate achieve 97.32% and 97.90% respectively.
Offline Handwritten Chinese Character Recognition Based on New Training Methodology

Weike Luo,Guangtao Zhai

DOI: https://doi.org/10.1007/978-981-10-8108-8_22

2018-01-01

Abstract:Deep learning based methods have been extensively used in Handwritten Chinese Character Recognition (HCCR) and significantly improved the recognition accuracy in recent years. Famous networks like GoogLeNet and deep residual network have been applied to this field and improved the recognition accuracy. While the structure of the neural network is crucial, the training methodology also plays an important role in deep learning based methods. In this paper, a new data generation method is proposed to increase the size of the training database. Chinese characters could be classified into different kinds of structures according to the radical components. Based on this, the proposed method segments the character images into sub-images and recombines them into new character samples. The generated database, including recombined characters and rotated characters, could improve the performance of current CNN models. We also apply the recently proposed and popular center loss function to further improve the recognition accuracy. Tested on ICDAR 2013 competition database, the proposed methods could achieve new state-of-the-art with a 97.53% recognition accuracy.
Deep Learning-Driven Approach for Handwritten Chinese Character Classification

Boris Kriuk,Fedor Kriuk

2024-03-27

Abstract:Handwritten character recognition (HCR) is a challenging problem for machine learning researchers. Unlike printed text data, handwritten character datasets have more variation due to human-introduced bias. With numerous unique character classes present, some data, such as Logographic Scripts or Sino-Korean character sequences, bring new complications to the HCR problem. The classification task on such datasets requires the model to learn high-complexity details of the images that share similar features. With recent advances in computational resource availability and further computer vision theory development, some research teams have effectively addressed the arising challenges. Although known for achieving high accuracy while keeping the number of parameters small, many common approaches are still not generalizable and use dataset-specific solutions to achieve better results. Due to complex structure, existing methods frequently prevent the solutions from gaining popularity. This paper proposes a highly scalable approach for detailed character image classification by introducing the model architecture, data preprocessing steps, and testing design instructions. We also perform experiments to compare the performance of our method with that of existing ones to show the improvements achieved.

Computer Vision and Pattern Recognition,Machine Learning
Deep Template Matching for Offline Handwritten Chinese Character Recognition

Zhiyuan Li,Min Jin,Qi Wu,Huaxiang Lu

DOI: https://doi.org/10.48550/arXiv.1811.06347

2018-11-15

Abstract:Just like its remarkable achievements in many computer vision tasks, the convolutional neural networks (CNN) provide an end-to-end solution in handwritten Chinese character recognition (HCCR) with great success. However, the process of learning discriminative features for image recognition is difficult in cases where little data is available. In this paper, we propose a novel method for learning siamese neural network which employ a special structure to predict the similarity between handwritten Chinese characters and template images. The optimization of siamese neural network can be treated as a simple binary classification problem. When the training process has been finished, the powerful discriminative features help us to generalize the predictive power not just to new data, but to entirely new classes that never appear in the training set. Experiments performed on the ICDAR-2013 offline HCCR datasets have shown that the proposed method has a very promising generalization ability to the new classes that never appear in the training set.

Computer Vision and Pattern Recognition
Writer Code Based Adaptation of Deep Neural Network for Offline Handwritten Chinese Text Recognition.

Zi-Rui Wang,Jun Du

DOI: https://doi.org/10.1109/icfhr.2016.0106

2016-01-01

Abstract:Recently, we propose deep neural network based hidden Markov models (DNN-HMMs)for offline handwritten Chinese text recognition. In this study, we design a novel writer code based adaptation on top of the DNN-HMM to further improve the accuracy via a customized recognizer. The writer adaptation is implemented by incorporating the new layers with the original input or hidden layers of the writer-independent DNN. These new layers are driven by the so-called writer code, which guides and adapts the DNN-based recognizer with the writer information. In the training stage, the writer-aware layers are jointly learned with the conventional DNN layers in an alternative manner. In the recognition stage, with the initial recognition results from the first-pass decoding with the writer-independent DNN, an unsupervised adaptation is performed to generate the writer code via the cross-entropy criterion for the subsequent. second-pass decoding. The experiments on the most challenging task of ICDAR 2013 Chinese handwriting competition show that our proposed adaptation approach can achieve consistent and significant improvements of recognition accuracy over a high-performance writer-independent DNN-HMM based recognizer across all 60 writers, yielding a relative character error rate reduction of 23.62% in average.
A New Hybrid-parameter Recurrent Neural Networks for Online Handwritten Chinese Character Recognition

Haiqing Ren,Weiqiang Wang

DOI: https://doi.org/10.48550/arXiv.1711.02809

2017-11-08

Computer Vision and Pattern Recognition

Abstract:The recurrent neural network (RNN) is appropriate for dealing with temporal sequences. In this paper, we present a deep RNN with new features and apply it for online handwritten Chinese character recognition. Compared with the existing RNN models, three innovations are involved in the proposed system. First, a new hidden layer function for RNN is proposed for learning temporal information better. we call it Memory Pool Unit (MPU). The proposed MPU has a simple architecture. Second, a new RNN architecture with hybrid parameter is presented, in order to increasing the expression capacity of RNN. The proposed hybrid-parameter RNN has parameter changes when calculating the iteration at temporal dimension. Third, we make a adaptation that all the outputs of each layer are stacked as the output of network. Stacked hidden layer states combine all the hidden layer states for increasing the expression capacity. Experiments are carried out on the IAHCC-UCAS2016 dataset and the CASIA-OLHWDB1.1 dataset. The experimental results show that the hybrid-parameter RNN obtain a better recognition performance with higher efficiency (fewer parameters and faster speed). And the proposed Memory Pool Unit is proved to be a simple hidden layer function and obtains a competitive recognition results.
Beyond Human Recognition: A CNN-based Framework for Handwritten Character Recognition

Li Chen,Song Wang,Wei Fan,Jun Sun,Satoshi Naoi

DOI: https://doi.org/10.1109/acpr.2015.7486592

2015-01-01

Abstract:Because of the various appearance (different writers, writing styles, noise, etc.), the handwritten character recognition is one of the most challenging task in pattern recognition. Through decades of research, the traditional method has reached its limit while the emergence of deep learning provides a new way to break this limit. In this paper, a CNN-based handwritten character recognition framework is proposed. In this framework, proper sample generation, training scheme and CNN network structure are employed according to the properties of handwritten characters. In the experiments, the proposed framework performed even better than human on handwritten digit (MNIST) and Chinese character (CASIA) recognition. The advantage of this framework is proved by these experimental results.
Deep Convolutional Neural Networks Based on Knowledge Distillation for Offline Handwritten Chinese Character Recognition

Hongli He,Zongnan Zhu,Zhuo Li,Yongping Dan

DOI: https://doi.org/10.20965/jaciii.2024.p0231

2024-03-20

Journal of Advanced Computational Intelligence and Intelligent Informatics

Abstract:Deep convolutional neural networks (DNNs) have achieved outstanding performance in this field. Meanwhile, handwritten Chinese character recognition (HCCR) is a challenging area of research in the field of computer vision. DNNs require a large number of parameters and high memory consumption. To address these issues, this paper proposes an approach based on an attention mechanism and knowledge distillation. The attention mechanism improves the feature extraction and the knowledge distillation reduces the number of parameters. The experimental results show that ResNet18 achieves a recognition accuracy of 97.63% on the HCCR dataset with 11.25 million parameters. Compared with other methods, this study improves the performance for HCCR.
Offline Handwritten Chinese Character Using Convolutional Neural Network: State-of-the-Art Methods

Yingna Zhong,Kauthar Mohd Daud,Ain Najiha Binti Mohamad Nor,Richard Adeyemi Ikuesan,Kohbalan Moorthy

DOI: https://doi.org/10.20965/jaciii.2023.p0567

2023-07-20

Journal of Advanced Computational Intelligence and Intelligent Informatics

Abstract:Given the presence of handwritten documents in human transactions, including email sorting, bank checks, and automating procedures, handwritten characters recognition (HCR) of documents has been invaluable to society. Handwritten Chinese characters (HCC) can be divided into offline and online categories. Online HCC recognition (HCCR) involves the trajectory movement of the pen tip for expressing linguistic content. In contrast, offline HCCR involves analyzing and categorizing the sample binary or grayscale images of characters. As recognition technology develops, academics’ interest in Chinese character recognition has continuously increased, as it significantly affects social and economic development. Recent development in this area is promising. However, the recognition accuracy of offline HCCR is still a sophisticated challenge owing to their complexity and variety of writing styles. With the advancement of deep learning, convolutional neural network (CNN)-based algorithms have demonstrated distinct benefits in offline HCCR and have achieved outstanding results. In this review, we aim to show the different HCCR methods for tackling the complexity and variability of offline HCC writing styles. This paper also reviews different activation functions used in offline HCCR and provides valuable assistance to new researchers in offline Chinese handwriting recognition by providing a succinct study of various methods for recognizing offline HCC.

Multi-Column Deep Neural Networks for Offline Handwritten Chinese Character Classification

SmartHandwriting: Handwritten Chinese Character Recognition with Smartwatch.

A New Hybrid-Parameter Recurrent Neural Network for Online Handwritten Chinese Character Recognition

Deep LSTM Networks for Online Chinese Handwriting Recognition

Deep Convolutional Neural Network Based Hidden Markov Model for Offline Handwritten Chinese Text Recognition

A Comprehensive Study of Hybrid Neural Network Hidden Markov Model for Offline Handwritten Chinese Text Recognition.

Multi-column Deep Neural Networks for Image Classification

Deep networks for online handwriting Chinese character recognition

Offline Handwritten Chinese Text Recognition with Convolutional Neural Networks

A Study of Designing Compact Classifiers Using Deep Neural Networks for Online Handwritten Chinese Character Recognition

Deep Matching Network for Handwritten Chinese Character Recognition

Improved Deep Convolutional Neural Network For Online Handwritten Chinese Character Recognition using Domain-Specific Knowledge

A Residual-Attention Offline Handwritten Chinese Text Recognition Based on Fully Convolutional Neural Networks.

Offline Handwritten Chinese Character Recognition Based on New Training Methodology

Deep Learning-Driven Approach for Handwritten Chinese Character Classification

Deep Template Matching for Offline Handwritten Chinese Character Recognition

Writer Code Based Adaptation of Deep Neural Network for Offline Handwritten Chinese Text Recognition.

A New Hybrid-parameter Recurrent Neural Networks for Online Handwritten Chinese Character Recognition

Beyond Human Recognition: A CNN-based Framework for Handwritten Character Recognition

Deep Convolutional Neural Networks Based on Knowledge Distillation for Offline Handwritten Chinese Character Recognition

Offline Handwritten Chinese Character Using Convolutional Neural Network: State-of-the-Art Methods