Abstract:Due to the polysemy and complexity of the Chinese language, Chinese machine reading comprehension has always been a challenging task. To improve the semantic understanding and robustness of Chinese machine reading comprehension models, we propose a model that utilizes adversarial training algorithms and Permuted Language Model (PERT). Firstly, we employ the PERT pre-training model to embed paragraphs and questions into vector space to obtain corresponding sequential representations. Secondly, we use a multi-head self-attention mechanism to extract key textual information from the sequence and employ a Bi-GRU network to semantically fuse the output feature vectors, aiming to learn deep semantic representations in the text. Finally, we introduce perturbations into the model training process. We achieve this by utilizing adversarial training algorithms such as Fast Gradient Method (FGM) and Projected Gradient Descent (PGD). These algorithms generate adversarial samples to enhance the model's robustness and stability when facing diverse inputs. We conducted comparative experiments on the publicly available Chinese reading comprehension datasets CMRC2018 and DRCD. The experimental results show that our proposed model has achieved significant improvements in both EM and F1-Score compared to the baseline model. To validate the model's generalization and robustness, we utilized ChatGPT to construct a scientific dataset that includes a large number of domain-specific terms, sentences with mixed Chinese and English, and complex comprehension tasks. Our model also performed remarkably well on the self-built dataset. In conclusion, the proposed model not only effectively enhances the understanding of semantic information in Chinese text but also demonstrates a certain level of generalization capability.

A cross-temporal contrastive disentangled model for ancient Chinese understanding

Automatic Translating Between Ancient Chinese and Contemporary Chinese with Limited Aligned Corpora.

Time-Aware Ancient Chinese Text Translation and Inference

Ancient-Modern Chinese Translation with a Large Training Dataset

Guwen-UNILM: Machine Translation Between Ancient and Modern Chinese Based on Pre-Trained Models

Comparative Analysis of Language Models for Linguistic Examination of Ancient Chinese Classics: A Case Study of Zuozhuan Corpus.

Can Large Language Model Comprehend Ancient Chinese? A Preliminary Test on ACLUE

That Slepen Al the Nyght with Open Ye! Cross-era Sequence Segmentation with Switch-memory

AnchiBERT: A Pre-Trained Model for Ancient ChineseLanguage Understanding and Generation

Automatic Transferring Between Ancient Chinese and Contemporary Chinese.

AnchiBERT: A Pre-Trained Model for Ancient Chinese Language Understanding and Generation

The Uncertainty-based Retrieval Framework for Ancient Chinese CWS and POS

Contrastive Learning between Classical and Modern Chinese for Classical Chinese Machine Reading Comprehension

CCDC: A Chinese-Centric Cross Domain Contrastive Learning Framework

AC-EVAL: Evaluating Ancient Chinese Language Understanding in Large Language Models

HC$^2$L: Hybrid and Cooperative Contrastive Learning for Cross-lingual Spoken Language Understanding

Improving Low-resource Reading Comprehension via Cross-lingual Transposition Rethinking

MULTI-LEVEL CONTRASTIVE LEARNING FOR CROSS-LINGUAL ALIGNMENT

Combining permuted language model and adversarial training for Chinese machine reading comprehension

Towards Effective Ancient Chinese Translation: Dataset, Model, and Evaluation