Abstract:Parsing, the task of identifying syntactic components, e.g., noun and verb phrases, in a sentence, is one of the fundamental tasks in natural language processing. Many natural language applications such as spoken-language understanding, machine translation, and information extraction, would benefit from, or even require, high accuracy parsing as a preprocessing step. Even though most state-of-the-art statistical parsers were initially constructed for parsing in English, most of them are not language-specific, in that they do not rely on properties of the language that are specific to English. Therefore, construction of a parser in a given language becomes a matter of retraining the statistical parameters with a Treebank in the corresponding language. The development of the Chinese treebank [Xia et al. 2000] spurred the construction of parsers for Chinese. However, Chinese as a language poses some unique problems for the development of a statistical parser, the most apparent being word segmentation. Since words in written Chinese are not delimited in the same way as in Western languages, the first problem that needs to be solved before an existing statistical method can be applied to Chinese is to identify the word boundaries. This is a step that is neglected by most pre-existing Chinese parsers, which assume that the input data has already been pre-segmented. This article describes a character-based statistical parser, which gives the best performance to-date on the Chinese treebank data. We augment an existing maximum entropy parser with transformation-based learning, creating a parser that can operate at the character level. We present experiments that show that our parser achieves results that are close to those achievable under perfect word segmentation conditions.

Iterative Annotation Transformation with Predict-Self Reestimation for Chinese Word Segmentation

Segment, Mask, and Predict: Augmenting Chinese Word Segmentation with Self-Supervision

Improving Chinese Word Segmentation Using Partially Annotated Sentences

Reducing Approximation and Estimation Errors for Chinese Lexical Processing with Heterogeneous Annotations

A New Error-driven Learning Approach for Chinese Word Segmentation

A Unified Model for Joint Chinese Word Segmentation and POS Tagging with Heterogeneous Annotation Corpora.

RethinkCWS: is Chinese Word Segmentation a Solved Task?

Adapting Conventional Chinese Word Segmenter for Segmenting Micro-blog Text: Combining Rule-based and Statistic-based Approaches.

Human-Computer Interactive Chinese Word Segmentation: an Adaptive Dirichlet Process Mixture Model Approach.

A realistic and robust model for Chinese word segmentation

Parsing-based Chinese word segmentation integrating morphological and syntactic information

A maximum-entropy chinese parser augmented by transformation-based learning

Joint Chinese Word Segmentation and POS Tagging on Heterogeneous Annotated Corpora with Multiple Task Learning.

Optimization of Chinese Word Segmentation in Named Entity Recognition and Word AIignment

Toward Fast and Accurate Neural Chinese Word Segmentation with Multi-Criteria Learning

CSeg& Tag1.0

Blending segmentation with tagging in Chinese language corpus processing

A Hybrid Approach to Chinese Word Segmentation around CRFs

A Joint Segmenting And Labeling Approach For Chinese Lexical Analysis

Unified Framework of Performing Chinese Word Segmentation and Part-Of-Speech Tagging

A Kalman Filter Based Human-Computer Interactive Word Segmentation System For Ancient Chinese Texts