Abstract:N-gram language modelling, a proven and effective method in NLP, is widely used to calculate the probability of a sentence in natural language. Language pattern is a linguistic level between word/character and sentence, which exists in pattern grammar. In this research, the approach of language model and language pattern are combined for the first time, and language patterns are studied by use of the N-gram model. Chinese function-word-centered patterns are extracted from the LCMC corpus, and aligned into pattern chains. The language model is trained from these chains to investigate the properties and distribution of Chinese function words, the interaction of content words and function words, and the interaction between patterns. The results indicate that there are approximately 10,000 function-word-centered patterns in the texts, which are distributed exponentially. This research summarizes the most common function-word-centered patterns and content-word-centered patterns, and discusses the interactions of patterns based on corpus data. The bigram language model of these patterns reflects the restrictions of function words. In addition, the research adopts an innovative method to visualize the interactions between patterns. This research fills the research gap between word/character and sentence and reveals basic Chinese pattern categories and the interactions between patterns, which makes a significant contribution to Chinese linguistic research, and improves the efficiency of NLP.

Text Classification Based on N-gram Language Model

Chinese Documents Classification Based on N-Grams

Categorizing Web Information on Subject with Statistical Language Modeling

Chinese Text Classification Without Word Segmentation

Hierarchical Classification of Chinese Documents Based on N-grams

Hierarchical classification of Chinese documents based onN-grams

Chinese Documents Categorization Based on N-gram Information

Comparing different text representation and feature selection methods on Chinese text classification using Character n-grams

Chinese Short-Text Categorization Based on the Key Classification Dictionary Words

N-gram Language Model for Chinese Function-word-centered Patterns

A Short Text Classification Method Based on N -Gram and CNN

Chinese Text Categorization Without Word Segmentation Using String Kernel

Improved N-gram Model Based on Ontology for Web Page Classification

Text Categorization with Lee Model

The Research of Chinese Automatic Word Segmentation System Based on N-Gram Statistical Model

Research on Text Classification Based on BERT-BiGRU Model

N-grams based feature selection and text representation for Chinese Text Classification

Feature Selection On Chinese Text Classification Using Character N-Grams

Chinese Text Classification Model Based on Deep Learning

Chinese News Text Classification Based on Machine Learning Algorithm