Abstract:Natural Language Understanding (NLU) and Natural Language Generation (NLG) are the general methods that support machine understanding of text content. They play a very important role in the text information processing system including recommendation and question and answer systems. There are many researches in the field of NLU such as Bag of words, N-Gram, and neural network language model. These models have achieved a good performance in NLU and NLG tasks. However, since they require lots of training data, it is difficult to obtain rich data in practical applications. Thus, pretraining becomes important. This paper proposes a semisupervised way to deal with math word problem (MWP) tasks using unsupervised pretraining and supervised tuning methods, which are based on the Unified pretrained Language Model (UniLM). The proposed model requires fewer training data than traditional models since it uses model parameters of tasks that have been learned before to initialize the model parameters of new tasks. In this way, old knowledge helps new models successfully perform new tasks from old experiences instead of from scratch. Moreover, in order to help the decoder make accurate predictions, we combine the advantages of AR and AE language models to support one-way, sequence-to-sequence, and two-way predictions. Experiments, carried out on MWP tasks with 20,000+ mathematical questions, show that the improved model outperforms the traditional models with a maximum accuracy of 79.57%. The impact of different experiment parameters is also studied in the paper and we found that a wrong arithmetic order leads to incorrect solution expression generation.

A Framework for Math Word Problem Solving Based on Pre-training Models and Spatial Optimization Strategies.

MWPToolkit: An Open-Source Framework for Deep Learning-Based Math Word Problem Solvers

MWPToolkit: an Open-Source Framework for Deep Learning-Based Math Word Problem Solvers.

Deep Learning in Automatic Math Word Problem Solvers

Generate & Rank: A Multi-task Framework for MathWord Problems

Modeling Intra-Relation in Math Word Problems with Different Functional Multi-Head Attentions

Techniques to Improve Neural Math Word Problem Solvers

Teacher-Student Networks with Multiple Decoders for Solving Math Word Problem

An Improved Math Word Problem (MWP) Model Using Unified Pretrained Language Model (UniLM) for Pretraining

Generate & Rank: A Multi-task Framework for Math Word Problems

Investigating Math Word Problems using Pretrained Multilingual Language Models

Math Word Problem Solving by Generating Linguistic Variants of Problem Statements

MathDQN: Solving Arithmetic Word Problems Via Deep Reinforcement Learning.

Tackling Math Word Problems with Fine-to-Coarse Abstracting and Reasoning

TM-generation model: a template-based method for automatically solving mathematical word problems

Goal selection and feedback for solving math word problems

Template-Based Math Word Problem Solvers with Recursive Neural Networks

Measuring Mathematical Problem Solving With the MATH Dataset

Enhancing Math Word Problem Solving Through Salient Clue Prioritization: A Joint Token-Phrase-Level Feature Integration Approach.

Solving Math Word Problems with Reexamination

DISK: Domain-constrained Instance Sketch for Math Word Problem Generation