Abstract:Abstract There exists no sentence boundary in most classical Chinese literature texts. Since it is difficult to read literature of this kind, experts in literature or linguistics would segment the sentence manually. This article explores the effectiveness of classical Chinese sentence segmentation method so as to provide a reference for classical Chinese punctuation. On the basis of the machine learning methods, we chose three components of machine learning, namely models, tagging schemes, and features, to compare the learning results. The models include conditional random field (CRF) models, long short term memory (LSTM) models, BiLSTM–CRF models, and three Bidirectional Encoder Representation from Transformers (BERT) models. There are five tagging schemes in this article and three features including the statistical feature, Guangyun, and Fanqie. Finally, the performance of the combined feature template is evaluated by ten-fold cross-validation on four classical Chinese texts in different genres. The SikuBERT model is proved to be the most effective model for sentence segmentation at present. Different tagging schemes and various features are introduced. The results show that 5-tag-J tagging schemes can improve performance. Statistical feature, as an important clue for classical Chinese sentence segmentation, is useful in related tasks, but Guangyun and Fanqie have little impact. Other important factors of sentence segmentation are genres and writing styles.

Chinese sentence segmentation based on SVM method

Chinese Word Segmentation Method Based on Dictionary and Frequency of the Words

Automatic sentence segmentation for classical Chinese: The Spring and Autumn Annals as an example

A SVM-Based Model for Chinese Functional Chunk Parsing.

A Sentence Segmentation Method for Ancient Chinese Texts Based on NNLM.

Increasing Accuracy of Chinese Segmentation with Strategy of Multi-step Processing

Parsing-based Chinese word segmentation integrating morphological and syntactic information

A Discriminative Latent Variable Chinese Segmenter with Hybrid Word/Character Information.

Chinese chunking with large margin method

Segmenting Long Sentence Pairs for Statistical Machine Translation

Analysis of Rapid Chinese Text Chunking Based on SVM

Chinese event descriptive clause splitting with structured SVMs

Sub-Sentence Division for Tree-Based Machine Translation.

Fast Neural Chinese Word Segmentation for Long Sentences

Joint Chinese Word Segmentation and Span-based Constituency Parsing

Detecting Syntactic Features of Translated Chinese

A Chinese Word Segmentation for Statistical Machine Translation

Improving Chinese Word Segmentation Using Partially Annotated Sentences

A Unified Model for Joint Chinese Word Segmentation and Dependency Parsing

A Metasynthetic Approach for Segmenting Handwritten Chinese Character Strings

Feature-rich Prepositional Phrase Boundary Identification based on SVM