Abstract:Recently, thanks to the successful application of the attention-based encoder-decoder framework, handwritten mathematical expression recognition (HMER) has achieved significant improvement. However, HMER is still a challenging task in the handwriting recognition area, which suffers from the ambiguity of handwritten symbols, the two-dimensional structure of mathematical expressions, and the lack of labeled data. In this paper, we attempt to improve the recognition performance and generalization ability of the existing state-of-the-art method from two perspectives: data augmentation and model design. We first propose a tree-based multi-level (including symbol level, sub-expression level, and image level) data augmentation strategy, which can generate many synthetic images. Then, we present a novel encoder-decoder hybrid model via tree-based mutual learning to fully utilize the complementarity between tree decoder and string decoder. Benefitting from our data augmentation strategy, we achieve 58.47%/57.82%/62.67% and 74.45% expression recognition accuracy respectively on the CROHME14/16/19 competition datasets and the OffRaSHME20 competition dataset. Moreover, tree-based data augmentation is a key technology to our champion system for the OffRaSHME20 competition. Our tree-based mutual learning method further improves the recognition accuracy to 61.63%/59.81%/64.38% and 75.68% on these datasets. Further quantitative and qualitative analyses also demonstrate the effectiveness and robustness of our proposed methods.

Handwritten Digit Recognition Via Active Belief Decision Trees

Dynamic hand gesture recognition using hidden Markov models

Improvements on EMG-based handwriting recognition with DTW algorithm.

Fast Online Handwritten Digits Recognition Based on Decision Tree

Research on Automatic Recognition of Unconstratined Handwritten Digits

A tree-BLSTM-based recognition system for online handwritten mathematical expressions

User-Independent Online Handwritten Digit Recognition

Online active learning of decision trees with evidential data

HandiText: Handwriting Recognition Based on Dynamic Characteristics with Incremental LSTM

Learning Sparse Feature Representations using Probabilistic Quadtrees and Deep Belief Nets

A Method of Handwriting Texts and Shapes Separation

A Bayesian model for recognizing handwritten mathematical expressions

A Classification Tree Method Based on Belief Entropy for Evidential Data

On-line Handwritten Mathematical Expression Recognition Method Based on Statistical and Semantic Analysis

Adaboost Based Handwritten/Printed Discrimination On Single Character

Bagging likelihood-based belief decision trees

Belief Entropy Tree and Random Forest: Learning from Data with Continuous Attributes and Evidential Labels

AdaBoost-based handwritten/printed discrimination on a single character

Learning Belief Regression Trees From Evidential Data

A tree-based model with branch parallel decoding for handwritten mathematical expression recognition

Tree-based Data Augmentation and Mutual Learning for Offline Handwritten Mathematical Expression Recognition