Abstract:Deep convolutional neuralnetworks have achieved fairly high accuracy for single online handwritten Chinese character recognition (SOLHCCR). However, in real application scenarios, users always write multiple characters to form a complete sentence, and previous contextual information holds significant potential for improving the accuracy, robustness and efficiency of recognition. In this work, we first propose a simple and straightforward model named the vanilla compositional network (VCN) by coupling convolutional neural network with a sequence modeling architecture (i.e., a recurrent neural network or Transformer), which exploits the handwritten character’s previous contextual information. Although VCN performs much better than the previous state-of-the-art SOLHCCR models, it is a two-stage architecture in nature. It suffers from high fragility when confronting with poorly written characters such as sloppy writing, and missing or broken strokes, due to relying heavily on contextual information. To improve the robustness of the OLHCCR model, we further propose a novel deep spatial & contextual information fusion network (DSCIFN). It utilizes an autoregresssive framework pre-trained on a large-scale sentence corpora as the backbone component, and highly integrates the spatial features of handwritten characters and their previous contextual information in a multi-layer fusion module. To verify the effectiveness of models, we reorganize a new form of online Chinese handwritten character with its previous context dataset, named OHCCC. Extensive experimental results demonstrate that DSCIFN achieves state-of-the-art performance and has increased strong robustness compared to VCN and previous SOLHCCR models. The in-depth empirical analysis and case study indicate that DSCIFN can significantly improve the efficiency of handwriting input because it does not need complete strokes to recognize a handwritten Chinese character precisely.

Instance GNN: A Learning Framework for Joint Symbol Segmentation and Recognition in Online Handwritten Diagrams

Deep Learning Based Hand Gesture Recognition in Complex Scenes

SpaceGTN: A Time-Agnostic Graph Transformer Network for Handwritten Diagram Recognition and Segmentation

Symbol detection in online handwritten graphics using Faster R-CNN

Characters as Graphs: Recognizing Online Handwritten Chinese Characters via Spatial Graph Convolutional Network

A general framework for the recognition of online handwritten graphics

DrawnNet: Offline Hand-Drawn Diagram Recognition Based on Keypoint Prediction of Aggregating Geometric Characteristics

A New Hybrid-Parameter Recurrent Neural Network for Online Handwritten Chinese Character Recognition

Joint stroke classification and text line grouping in online handwritten documents with edge pooling attention networks

Sentence-level Online Handwritten Chinese Character Recognition

Drawing and Recognizing Chinese Characters with Recurrent Neural Network

GMU: A Novel RNN Neuron and Its Application to Handwriting Recognition.

On-line Handwritten Flowchart Recognition Based on Logical Structure and Graph Grammar

Arrow R-CNN for handwritten diagram recognition

Learning Spatial-Semantic Context with Fully Convolutional Recurrent Network for Online Handwritten Chinese Text Recognition

First Experiments On A New Online Handwritten Flowchart Database

A Comprehensive Study of Hybrid Neural Network Hidden Markov Model for Offline Handwritten Chinese Text Recognition.

Fast and Robust Online Handwritten Chinese Character Recognition with Deep Spatial and Contextual Information Fusion Network

DyGAT: Dynamic Stroke Classification of Online Handwritten Documents and Sketches

Deep LSTM Networks for Online Chinese Handwriting Recognition

Multi-layer Recurrent Neural Network Based Offline Arabic Handwriting Recognition