KwaiYiiMath: Technical Report

Jiayi Fu,Lei Lin,Xiaoyang Gao,Pengli Liu,Zhengzong Chen,Zhirui Yang,Shengnan Zhang,Xue Zheng,Yan Li,Yuliang Liu,Xucheng Ye,Yiqiao Liao,Chao Liao,Bin Chen,Chengru Song,Junchen Wan,Zijia Lin,Fuzheng Zhang,Zhongyuan Wang,Di Zhang,Kun Gai

DOI: https://doi.org/10.48550/arXiv.2310.07488

2023-10-11

Computation and Language

Abstract:Recent advancements in large language models (LLMs) have demonstrated remarkable abilities in handling a variety of natural language processing (NLP) downstream tasks, even on mathematical tasks requiring multi-step reasoning. In this report, we introduce the KwaiYiiMath which enhances the mathematical reasoning abilities of KwaiYiiBase1, by applying Supervised Fine-Tuning (SFT) and Reinforced Learning from Human Feedback (RLHF), including on both English and Chinese mathematical tasks. Meanwhile, we also constructed a small-scale Chinese primary school mathematics test set (named KMath), consisting of 188 examples to evaluate the correctness of the problem-solving process generated by the models. Empirical studies demonstrate that KwaiYiiMath can achieve state-of-the-art (SOTA) performance on GSM8k, CMath, and KMath compared with the similar size models, respectively.