Online Handwritten Mathematical Expressions Recognition System Using Fuzzy Neural Network

E. Naderan

DOI: https://doi.org/10.48550/arXiv.1707.03088

2017-07-11

Computer Vision and Pattern Recognition

Abstract:The article describes developed information technology for online recognition of handwritten mathematical expressions that based on proposed approaches to handwritten symbols recognition and structural analysis.

What problem does this paper attempt to address?

An on-line handwritten arithmetic expression recognition system

Han Jiang,Liu Chang-song,Shi Yang Li,Zou Yan-ming

DOI: https://doi.org/10.3724/SP.J.1146.2009.00549

2010-01-01

Abstract:In this paper a complete on-line handwritten arithmetic expression recognition system is introduced. During the process of stroke grouping and symbol recognition, pre-processing strategy and searching procedure are combined to optimize the confidence function. A novel structural analysis algorithm based on symbol ranking is described, and recognition results are stored in trinary trees as the data structure. Experiments show that relatively neatly written expressions with fairly complex structures can be analysed by this algorithm with a high correction rate.
A Novel Method for Recognition of Persian Alphabet by Using Fuzzy Neural Network.

Mohammad Mehdi Motahari Kia,Jafar A. Alzubi,Mehdi Gheisari,Xiaobo Zhang,Mohamadtaghi Rahimi,Yongrui Qin

DOI: https://doi.org/10.1109/ACCESS.2018.2881050

IF: 3.9

2018-01-01

IEEE Access

Abstract:This paper presents a system that can recognize handwritten words expressed using broken letters of the Persian alphabet. The proposed system can be used for most activities related to the gathering of public information. Statistical features of the separated/broken letters are employed in the system. Each letter is recognized using interconnected fuzzy neural network. The advantages of this method include high precision owing to the strength of the neural network algorithm and the possibility of extending dataset instance codes in a simple manner. At last, an evaluation for the proposed method is provided experimentally.
Offline Handwritten Mathematical Expression Evaluator Using Convolutional Neural Network

Amit Choudhary,Savita Ahlawat,Harsh Gupta,Aniruddha Bhandari,Ankur Dhall,Manish Kumar

DOI: https://doi.org/10.1007/978-981-15-5148-2_47

2020-07-31

Abstract:Recognition of Offline Handwritten Mathematical Expression (HME) is a complicated task in the field of computer vision. The proposed method in this paper follows three steps: segmentation, recognition and evaluation of the HME image (which may include multiple mathematical expressions and linear equations). The segmentation of symbols from image incorporates a novel pre-contour filtration technique to remove distortions from segmented symbols. Then, recognition of segmented symbols is done using Convolutional Neural Network which is trained on an augmented dataset prepared from EMNIST and custom-built dataset giving an accuracy of 97% in recognizing the symbols correctly. Finally, the expressions/equations are evaluated by tokenizing, converting into postfix expressions and then solving using a custom-built parser.
Watch, Attend and Parse: an End-to-end Neural Network Based Approach to Handwritten Mathematical Expression Recognition.

Jianshu Zhang,Jun Du,Shiliang Zhang,Dan Liu,Yulong Hu,Jinshui Hu,Si Wei,Lirong Dai

DOI: https://doi.org/10.1016/j.patcog.2017.06.017

IF: 8

2017-01-01

Pattern Recognition

Abstract:•A novel neural network based approach to handwritten mathematical expression recognition.•An end-to-end encoder-decoder framework to alleviate the problem caused by an explicit symbol segmentation and the computational demands of employing a mathematical expression grammar.•The deep fully convolutional neural network as the encoder.•The coverage-based attention model to incorporate the attention history.•Attention visualization to show the link between the input image and output symbol sequence in latex format.•To the best of our knowledge, achieving the best published expression recognition accuracy on CROHME 2014 competition set by only using the official training data.
A Hybrid Handwritten Digits Recognition System Based on Neural Networks and Fuzzy Logic

W Lu,BX Shi,ZJ Li

DOI: https://doi.org/10.1109/icsmc.1996.569810

1996-01-01

Abstract:A hybrid handwriting recognition system based on neural networks and fuzzy logic is proposed. The system consists of two stages. In the first stage, a Hamming neural network is used to extract local features from a pattern, and based on the feature maps, a fuzzy logic recognizer is adopted to do the recognition. In the second stage, the horizontal/vertical connected component features are extracted, the recognition task is also performed by a fuzzy logic recognizer. Experiments show that the performance of the hybrid system is better than either of both stages. It has very high recognition speed and large ability to deal with distortion and shift variations in handwriting characters.
HANDWRITTEN MATHEMATICAL EQUATION SOLVER

Rajwardhan Shinde,Onkar Dherange,Rahul Gavhane,Hemant Koul,Nilam Patil,,,,,

DOI: https://doi.org/10.33564/ijeast.2022.v06i10.018

2022-02-01

International Journal of Engineering Applied Sciences and Technology

Abstract:With recent developments in Artificial intelligence and deep learning every major field which is using computers for any type of work is trying to ease the work using deep learning methods. Deep learning is used in a wide range of fields due to its diverse range of applications like health, sports, robotics, education, etc. In deep learning, a Convolutional neural network (CNN) is being used in image classification, pattern recognition, Text classification, face recognition, live monitoring systems, handwriting recognition, Digit recognition, etc. In this paper, we propose a system for educational use where the recognition and solving process of mathematical equations will be done by machine. In this system for recognition of equations, we use a Convolutional neural network (CNN) model. The proposed system can recognize and solve mathematical equations with basic operations (-,+,/,*) of multiple digits as well as polynomial equations. The model is trained with Modified National Institute of Standards and Technology (MNIST) dataset as well as a manually prepared dataset of operator symbols (“-”,”+”, “/”, “*”, “(“, “)” ). Further, the system uses the RNN model to solve the recognized operations.
On-line Handwritten Mathematical Expression Recognition Method Based on Statistical and Semantic Analysis

Yang Hu,Liangrui Peng,Yejun Tang

DOI: https://doi.org/10.1109/DAS.2014.47

2014-01-01

Abstract:Recognition of handwritten mathematical expressions (HMEs) has become a cutting edge research topic recently, as there are increasingly needs for pen-inputting applications. In this paper, we presented a novel framework to analyse HME layout and semantic information. This framework includes three steps, namely symbol segmentation, symbol recognition and semantic relationship analysis. For symbol segmentation, a decomposition on strokes is operated, then dynamic programming is adopted to find the paths corresponding to the best segmentation manner and reduce the stroke searching complexity. For symbol recognition, spatial geometry and directional element features are classified by a Gaussian Mixture Model learnt through Expectation-Maximization algorithm. At last, in the semantic relationship analysis module, a ternary tree is utilized to to store the ranked symbols through calculating the operator priorities. The motivation for our work comes from the apparent difference in writing styles across western and Chinese populations. Our results are reasonable and show promise on the private dataset.
A System for Recognizing Online Handwritten Mathematical Expressions and Improvement of Structure Analysis

Anh Duc Le,Truyen Van PHAN,Masaki Nakagawa,Truyen Van Phan

DOI: https://doi.org/10.1109/das.2014.52

2014-04-01

Abstract:This paper presents a system for recognizing online handwritten mathematical expressions (MEs) and improvement of structure analysis. We represent MEs in Context Free Grammars (CFGs) and employ the Cocke-Younger-Kasami (CYK) algorithm to parse 2D structure of on-line handwritten MEs and select the best interpretation in terms of symbol segmentation, recognition and structure analysis. We propose a method to learn structural relations from training patterns without any heuristic decisions by using two SVM models. We employ stroke order to reduce the complexity of the parsing algorithm. Moreover, we revise structure analysis. Even though CFG does not resolve ambiguities in some cases, our method still gives users a list of candidates that contain expecting result. We evaluate our method in the CROHME 2013 database and demonstrate the improvement of our system in recognition rate as well as processing time.
Offline handwritten mathematical symbol recognition utilising deep learning

Azadeh Nazemi,Niloofar Tavakolian,Donal Fitzpatrick,Chandrik a Fernando,Ching Y. Suen

DOI: https://doi.org/10.48550/arXiv.1910.07395

2019-10-16

Computer Vision and Pattern Recognition

Abstract:This paper describes an approach for offline recognition of handwritten mathematical symbols. The process of symbol recognition in this paper includes symbol segmentation and accurate classification for over 300 classes. Many multidimensional mathematical symbols need both horizontal and vertical projection to be segmented. However, some symbols do not permit to be projected and stop segmentation, such as the root symbol. Besides, many mathematical symbols are structurally similar, specifically in handwritten such as 0 and null. There are more than 300 Mathematical symbols. Therefore, designing an accurate classifier for more than 300 classes is required. This paper initially addresses the issue regarding segmentation using Simple Linear Iterative Clustering (SLIC). Experimental results indicate that the accuracy of the designed kNN classifier is 84% for salient, 57% Histogram of Oriented Gradient (HOG), 53% for Linear Binary Pattern (LBP) and finally 43% for pixel intensity of raw image for 66 classes. 87 classes using modified LeNet represents 90% accuracy. Finally, for 101 classes, SqueezeNet ac
A tree-BLSTM-based recognition system for online handwritten mathematical expressions

Ting Zhang,Harold Mouchère,Christian Viard-Gaudin

DOI: https://doi.org/10.1007/s00521-018-3817-2

2018-10-19

Neural Computing and Applications

Abstract:Long short-term memory networks (LSTM) achieve great success in temporal dependency modeling for chain-structured data, such as texts and speeches.An extension toward more complex data structures as encountered in 2D graphic languages is proposed in this work.Specifically, we address the problem of handwritten mathematical expression recognition, using a tree-based BLSTM architecture allowing the direct labeling of nodes (symbol) and edges (relationship) from a graph modeling the input strokes.One major difference with the traditional approaches is that there is no explicit segmentation, recognition and layout extraction steps but a unique trainable system that produces directly a stroke label graph describing a mathematical expression. The proposed system,considering no grammar, achieves competitive results in online math expression recognition domain.

computer science, artificial intelligence
HANDWRITTEN MATHEMATICAL EXPRESSION RECOGNITION AND GRADING SYSTEM

Pratik U Patil,Zaid Z,Kajol N Khot,Sikandar N,Seema G

DOI: https://doi.org/10.33564/ijeast.2021.v06i03.050

2021-07-01

International Journal of Engineering Applied Sciences and Technology

Abstract:Handwritten Mathematical Expression recognition and grading system is a challenging task in the field of pattern recognition. A lot of researchers have already worked on Handwritten Mathematical Expression recognition and have used various classifiers. In past, Convolutional Neural Network, also called CNN, has been highly used for recognizing patterns. In this paper, We propose an idea to recognize HME and evaluate offline using CNN for classification. The steps included are, first the worksheet is scanned and is sent to the work-spaces detection module where it will return all the rectangular work-spaces from the given worksheet, then the workspaces are sent to the line extraction module to extract all the lines. The extracted lines are then passed to the character segmentation module where it will segment the character and then characters will be classified using deep learning model DCCNN. Finally, the evaluation module will assess the line and draw a green/red bounding box depending on whether the line is correct or not.
Deep Learning Architecture for Off-Line Recognition of Handwritten Math Symbols

Kawther Khazri Ayeb,Yosra Meguebli,Afef Kacem Echi

DOI: https://doi.org/10.1007/978-3-030-71804-6_15

2021-01-01

Pattern Recognition and Artificial Intelligence

Abstract:Real scientific challenge, handwritten math formula recognition is an attractive field of pattern recognition leading to practical applications. Hundreds of alphanumeric and math symbols need to be recognized, many are so similar in appearance that some use of context is necessary for disambiguation. Analysis of the spatial relationships between symbols is challenging. In this work, we focus on handwritten math symbols and propose to recognize them by a deep learning approach. The symbol images, used for train, validation, and test are generated from Competition on Recognition of Online Handwritten Mathematical Expressions dataset (CROHME) 2019’s online patterns of mathematical symbols. As the large dataset is crucial for the performance of the deep learning model and it is labor-intensive to obtain a large amount of labeled data in real applications, we first augmented the database. Standing on the transfer learning technique, we then tested and compared several pre-trained Convolutional Neural networks (CNNs) like VGGNet, SqueezeNet, DenseNet, and Xception network and we tuned them to better fit our data. An accurate classification of 91.88% (train), 88.82% (validation), and 83.68% (test) for 101 classes is achieved, using only off-line features of the symbols.
Handwritten Farsi Character Recognition using Artificial Neural Network

Reza Gharoie Ahangar,Mohammad Farajpoor Ahangar

DOI: https://doi.org/10.48550/arXiv.0908.4386

2009-08-30

Abstract:Neural Networks are being used for character recognition from last many years but most of the work was confined to English character recognition. Till date, a very little work has been reported for Handwritten Farsi Character recognition. In this paper, we have made an attempt to recognize handwritten Farsi characters by using a multilayer perceptron with one hidden layer. The error backpropagation algorithm has been used to train the MLP network. In addition, an analysis has been carried out to determine the number of hidden nodes to achieve high performance of backpropagation network in the recognition of handwritten Farsi characters. The system has been trained using several different forms of handwriting provided by both male and female participants of different age groups. Finally, this rigorous training results an automatic HCR system using MLP network. In this work, the experiments were carried out on two hundred fifty samples of five writers. The results showed that the MLP networks trained by the error backpropagation algorithm are superior in recognition accuracy and memory usage. The result indicates that the backpropagation network provides good recognition accuracy of more than 80% of handwritten Farsi characters.

Computer Vision and Pattern Recognition
Unwavering Mathematical Equation Deciphering from the Paperwork Imagery

Dr. J. Sreerambabu,Mr. N. Santhosh,Mr. D. Rajkumar,Ms. V. N. Nevedha

DOI: https://doi.org/10.22214/ijraset.2023.54989

2023-08-31

International Journal for Research in Applied Science and Engineering Technology

Abstract:Abstract: The paper proposes a new approach for identifying equations in document images, with an emphasis on handwritten equations. The proposed strategy makes use of Python-based machine learning algorithms, and for the user interface, HTML, CSS, and JS. The document image is transformed into a machine-readable format during the pre-processing stage utilizing optical character recognition (OCR). Then, image-improving methods like smoothing, thresholding, and morphological procedures are used. A deep neural network architecture is used to accurately identify math expressions in the pre-processed photos. Convolutional and recurrent layers make up this layout, which was developed utilizing machine learning techniques
Learning symbol relation tree for online mathematical expression recognition

Thanh-Nghia Truong,Hung Tuan Nguyen,Cuong Tuan Nguyen,Masaki Nakagawa

DOI: https://doi.org/10.48550/arXiv.2105.06084

2021-05-13

Abstract:This paper proposes a method for recognizing online handwritten mathematical expressions (OnHME) by building a symbol relation tree (SRT) directly from a sequence of strokes. A bidirectional recurrent neural network learns from multiple derived paths of SRT to predict both symbols and spatial relations between symbols using global context. The recognition system has two parts: a temporal classifier and a tree connector. The temporal classifier produces an SRT by recognizing an OnHME pattern. The tree connector splits the SRT into several sub-SRTs. The final SRT is formed by looking up the best combination among those sub-SRTs. Besides, we adopt a tree sorting method to deal with various stroke orders. Recognition experiments indicate that the proposed OnHME recognition system is competitive to other methods. The recognition system achieves 44.12% and 41.76% expression recognition rates on the Competition on Recognition of Online Handwritten Mathematical Expressions (CROHME) 2014 and 2016 testing sets.

Computer Vision and Pattern Recognition,Machine Learning
Symbol Segmentation and Recognition in Online Handwritten Formulas

YU Jin,DING Xiao-qing

DOI: https://doi.org/10.3969/j.issn.1002-8692.2007.z1.051

2007-01-01

Abstract:The arithmetic is concerned with the first processing stage of the online formula recognition system:the symbol seg- mentation and recognition task in the context of the formulas.It uses additional knowledge to merge the stroke into blocks and applies dynamic programming to find the best segmentation path.The recognition results obtained by different writers demonstrate that the performance of the approach is relatively good.
Artificial neural networks and fuzzy logic for recognizing alphabet characters and mathematical symbols

Giuseppe Airò Farulla,Tiziana Armano,Anna Capietto,Nadir Murru,Rosaria Rossini

DOI: https://doi.org/10.1007/978-3-319-41264-1_1

2016-07-06

Abstract:Optical Character Recognition software (OCR) are important tools for obtaining accessible texts. We propose the use of artificial neural networks (ANN) in order to develop pattern recognition algorithms capable of recognizing both normal texts and formulae. We present an original improvement of the backpropagation algorithm. Moreover, we describe a novel image segmentation algorithm that exploits fuzzy logic for separating touching characters.

Neural and Evolutionary Computing,Machine Learning
HANDWRITTEN CHARACTER RECOGNITION USING CONVOLUTIONAL NEURAL NETWORKS

D.J. Samatha Naidu,T. Mahammad Rafi

DOI: https://doi.org/10.47760/ijcsmc.2021.v10i08.007

2021-08-30

International Journal of Computer Science and Mobile Computing

Abstract:Handwritten character Recognition is one of the active area of research where deep neural networks are been utilized. Handwritten character Recognition is a challenging task because of many reasons. The Primary reason is different people have different styles of handwriting. The secondary reason is there are lot of characters like capital letters, small letters & special symbols. In existing were immense research going on the field of handwritten character recognition system has been design using fuzzy logic and created on VLSI(very large scale integrated)structure. To Recognize the tamil characters they have use neural networks with the Kohonen self-organizing map(SOM) which is an unsupervised neural networks. In proposed system this project design a image segmentation based hand written character recognition system. The convolutional neural network is the current state of neural network which has wide application in fields like image, video recognition. The system easily identify or easily recognize text in English languages and letters, digits. By using Open cv for performing image processing and having tensor flow for training the neural network. To develop this concept proposing the innovative method for offline handwritten characters. detection using deep neural networks using python programming language.
Offline handwritten mathematical expression recognition with graph encoder and transformer decoder

Jia-Man Tang,Hong-Yu Guo,Jin-Wen Wu,Fei Yin,Lin-Lin Huang

DOI: https://doi.org/10.1016/j.patcog.2023.110155

IF: 8

2023-11-29

Pattern Recognition

Abstract:Handwritten mathematical expression recognition (HMER) has attracted extensive attention. Despite the significant progress achieved in recent years attributed to the development of deep learning approaches, HMER remains a challenge due to the complex spatial structure and variable writing styles. Encoder–decoder models with attention mechanism, which treats HMER as an image-to-sequence (i.e. LaTeX) generation task, have boosted the accuracy, but suffer from low interpretability in that the symbols are not segmented explicitly. Symbol segmentation is desired for facilitating post-processing and human interaction in real applications. In this paper, we formulate the mathematical expression as a graph and propose a Graph-Encoder-Transformer-Decoder (GETD) approach for HMER. For constructing the graph from input image, candidate symbols are first detected using an object detector and represented as the nodes of a graph, called symbol graph, and the edges of the graph encodes the between-symbol relationship. The spatial information is aggregated in a graph neural network (GNN), and a Transformer-based decoder is used to identify the symbol classes and structure from the graph. Experiments on public datasets demonstrate that our GETD model achieves competitive expression recognition performance while offering good interpretability compared with previous methods.

computer science, artificial intelligence,engineering, electrical & electronic
Synthetic Data Generation and Shuffled Multi-Round Training Based Offline Handwritten Mathematical Expression Recognition

Lan-Fang Dong,Han-Chao Liu,Xin-Ming Zhang

DOI: https://doi.org/10.1007/s11390-021-0722-4

IF: 1.871

2022-12-30

Journal of Computer Science and Technology

Abstract:Offline handwritten mathematical expression recognition is a challenging optical character recognition (OCR) task due to various ambiguities of handwritten symbols and complicated two-dimensional structures. Recent work in this area usually constructs deeper and deeper neural networks trained with end-to-end approaches to improve the performance. However, the higher the complexity of the network, the more the computing resources and time required. To improve the performance without more computing requirements, we concentrate on the training data and the training strategy in this paper. We propose a data augmentation method which can generate synthetic samples with new LaTeX notations by only using the official training data of CROHME. Moreover, we propose a novel training strategy called Shuffled Multi-Round Training (SMRT) to regularize the model. With the generated data and the shuffled multi-round training strategy, we achieve the state-of-the-art result in expression accuracy, i.e., 59.74% and 61.57% on CROHME 2014 and 2016, respectively, by using attention-based encoder-decoder models for offline handwritten mathematical expression recognition.

computer science, software engineering, hardware & architecture

Online Handwritten Mathematical Expressions Recognition System Using Fuzzy Neural Network

An on-line handwritten arithmetic expression recognition system

A Novel Method for Recognition of Persian Alphabet by Using Fuzzy Neural Network.

Offline Handwritten Mathematical Expression Evaluator Using Convolutional Neural Network

Watch, Attend and Parse: an End-to-end Neural Network Based Approach to Handwritten Mathematical Expression Recognition.

A Hybrid Handwritten Digits Recognition System Based on Neural Networks and Fuzzy Logic

HANDWRITTEN MATHEMATICAL EQUATION SOLVER

On-line Handwritten Mathematical Expression Recognition Method Based on Statistical and Semantic Analysis

A System for Recognizing Online Handwritten Mathematical Expressions and Improvement of Structure Analysis

Offline handwritten mathematical symbol recognition utilising deep learning

A tree-BLSTM-based recognition system for online handwritten mathematical expressions

HANDWRITTEN MATHEMATICAL EXPRESSION RECOGNITION AND GRADING SYSTEM

Deep Learning Architecture for Off-Line Recognition of Handwritten Math Symbols

Handwritten Farsi Character Recognition using Artificial Neural Network

Unwavering Mathematical Equation Deciphering from the Paperwork Imagery

Learning symbol relation tree for online mathematical expression recognition

Symbol Segmentation and Recognition in Online Handwritten Formulas

Artificial neural networks and fuzzy logic for recognizing alphabet characters and mathematical symbols

HANDWRITTEN CHARACTER RECOGNITION USING CONVOLUTIONAL NEURAL NETWORKS

Offline handwritten mathematical expression recognition with graph encoder and transformer decoder

Synthetic Data Generation and Shuffled Multi-Round Training Based Offline Handwritten Mathematical Expression Recognition