Applying Class Triggers in Chinese Pos Tagging Based on Maximum Entropy Model

Y Zhao,XL Wang,BQ Liu,Y Guan
DOI: https://doi.org/10.1109/icmlc.2004.1382038
2004-01-01
Abstract:A method of applying class triggers in Chinese POS tagging based on maximum entropy model is proposed in this paper. First of all, feature template of "word-> word/tag" is used to extract the triggers from corpus and the triggers that we extracted are added into the maximum entropy model as a new kind of feature. Then, the average mutual information is applied to make feature selection and the semantic lexicon is used to build class triggers to overcome sparseness problem. Meanwhile, a solution based on experience to deal with over-fitting problem in model training is presented. Finally, the performance of the system is evaluated on a manually annotated POS tag corpus. The experiment demonstrates that the method can provide increase of accuracy of POS tagging from 94% to 96%, compared our new model with HMM model that is smoothed by absolute smoothing.
What problem does this paper attempt to address?