Statistical Overview Of Wordnet From 1.6 To 2.0

Jiangsheng Yu,Zhenshan Wen,Yang Liu,Zhihui Jin

2003-01-01

Abstract:We defined several discrete random variables and made their statistical comparisons between different versions of WordNet, by which the macroscopical evolution of WordNet from 1.6 to 2.0 is explored. And at the same time, the examples of extreme data will be enumerated during the experimental analysis.

What problem does this paper attempt to address?

New Word Identification in Social Network Text Based on Time Series Information

Meng Wang,Lanfen Lin,Feng Wang

DOI: https://doi.org/10.1109/cscwd.2014.6846904

2014-01-01

Abstract:Different from the languages widely used in western countries such as English or French, there are no spaces between words in Chinese language, and a segmentation of the texts is necessary before other superior processes. New word identification is an important problem in the segmentation process, especially when the segmentation targets are social network texts which have more abbreviated words or other non-standard representations. Several methods have been proposed to detect Chinese new words. Most of these methods take the corpus as a static set and they don't consider the time domain information. Different from these studies, we regard our social network corpus as a text series spreading along the time line and design a new kind of features named dynamic features which can reflect the temporal variety of the string's statistical features. The experimental results on the dataset crawled from the biggest microblogging application in China show that this method can significantly improve the effect of Chinese new word identification.
The Natural Selection of Words: Finding the Features of Fitness

Peter D. Turney,Saif M. Mohammad

DOI: https://doi.org/10.1371/journal.pone.0211512

2019-08-20

Abstract:We introduce a dataset for studying the evolution of words, constructed from WordNet and the Google Books Ngram Corpus. The dataset tracks the evolution of 4,000 synonym sets (synsets), containing 9,000 English words, from 1800 AD to 2000 AD. We present a supervised learning algorithm that is able to predict the future leader of a synset: the word in the synset that will have the highest frequency. The algorithm uses features based on a word's length, the characters in the word, and the historical frequencies of the word. It can predict change of leadership (including the identity of the new leader) fifty years in the future, with an F-score considerably above random guessing. Analysis of the learned models provides insight into the causes of change in the leader of a synset. The algorithm confirms observations linguists have made, such as the trend to replace the -ise suffix with -ize, the rivalry between the -ity and -ness suffixes, and the struggle between economy (shorter words are easier to remember and to write) and clarity (longer words are more distinctive and less likely to be confused with one another). The results indicate that integration of the Google Books Ngram Corpus with WordNet has significant potential for improving our understanding of how language evolves.

Computation and Language,Machine Learning
How Does Language Change As a Lexical Network? an Investigation Based on Written Chinese Word Co-Occurrence Networks

Heng Chen,Xinying Chen,Haitao Liu

DOI: https://doi.org/10.1371/journal.pone.0192545

IF: 3.7

2018-01-01

PLoS ONE

Abstract:Language is a complex adaptive system, but how does it change? For investigating this process, four diachronic Chinese word co-occurrence networks have been built based on texts that were written during the last 2,000 years. By comparing the network indicators that are associated with the hierarchical features in language networks, we learn that the hierarchy of Chinese lexical networks has indeed evolved over time at three different levels. The connections of words at the micro level are continually weakening; the number of words in the meso-level communities has increased significantly; and the network is expanding at the macro level. This means that more and more words tend to be connected to medium-central words and form different communities. Meanwhile, fewer high-central words link these communities into a highly efficient small-world network. Understanding this process may be crucial for understanding the increasing structural complexity of the language system.
Wordnet Based Comparison Of Language Variation: A Study Based On Ccd And Cwn

Jia-Fei Hong,Chu-Ren Huang,Yang Liu

2005-01-01

Abstract:This paper will deal with the lexica of comparing the Chinese Concept Dictionary (CCD) with the Chinese WordNet (CWN) by WordNet. CCD is a WordNet-like semantic lexicon that developed by the Institute of Computational Linguistics, Peking University. And CWN is a bilingual wordnet by linking to the SUMO ontology that developed by Academia Sinica Bilingual Ontological WordNet. In this paper, we will base on WordNet database to show several situations for both CCD and CWN, such as: the same translation for them, zero translation only for CCD or CWN, and unique translation only for CCD or CWN. Then, through these analyses, we could find out the unique usage of English translating for traditional Chinese Characters or simplified Chinese characters.
Polish and English wordnets -- statistical analysis of interconnected networks

Maksymilian Bujok,Piotr Fronczak,Agata Fronczak

DOI: https://doi.org/10.48550/arXiv.1404.1890

2014-04-07

Computation and Language

Abstract:Wordnets are semantic networks containing nouns, verbs, adjectives, and adverbs organized according to linguistic principles, by means of semantic relations. In this work, we adopt a complex network perspective to perform a comparative analysis of the English and Polish wordnets. We determine their similarities and show that the networks exhibit some of the typical characteristics observed in other real-world networks. We analyse interlingual relations between both wordnets and deliberate over the problem of mapping the Polish lexicon onto the English one.
Empirical Analysis on a Keyword-Based Semantic System

Zi-Ke Zhang,Linyuan Lü,Jian-Guo Liu,Tao Zhou

DOI: https://doi.org/10.1140/epjb/e2008-00453-9

2008-01-01

The European Physical Journal B

Abstract:Keywords in scientific articles have found their significance in information filtering and classification. In this article, we empirically investigated statistical characteristics and evolutionary properties of keywords in a very famous journal, namely Proceedings of the National Academy of Science of the United States of America (PNAS), including frequency distribution, temporal scaling behavior, and decay factor. The empirical results indicate that the keyword frequency in PNAS approximately follows a Zipf's law with exponent 0.86. In addition, there is a power-low correlation between the cumulative number of distinct keywords and the cumulative number of keyword occurrences. Extensive empirical analysis on some other journals' data is also presented, with decaying trends of most popular keywords being monitored. Interestingly, top journals from various subjects share very similar decaying tendency, while the journals of low impact factors exhibit completely different behavior. Those empirical characters may shed some light on the in-depth understanding of semantic evolutionary behaviors. In addition, the analysis of keyword-based system is helpful for the design of corresponding recommender systems.
The Global Organization of the English Lexicon and Its Evolution

Mieko Ogura,William S-Y. Wang

DOI: https://doi.org/10.1075/cilt.324.04ogu

2012-01-01

Abstract:We present a quantitative study of the semantic network of the set of nouns and verbs of WordNet, which is a systematic representation of the Present-day English lexicon based on psycholinguistic considerations, and A Thesaurus of Old English to understand the evolution of the global organization of the English lexicon. We demonstrate that whereas the semantic network is dominated by the hypernymy tree, which works as the skeleton of the set of nouns and verbs, the inclusion of polysemy produces a drastic global reorganization of the semantic structure, that is, it is converted into a small world, where all meanings are closer to each other. We then show that the words with higher frequency and therefore with higher number of meanings construct the higher level of the hypernymy tree within each lexical category. This architecture is robust through the times, forming the basis of the smallworld network. We also suggest that the small-world topology of the brain has enhanced the small-world configuration of semantic structure.
Statistical Laws Governing Fluctuations in Word Use from Word Birth to Word Death

Alexander M. Petersen,Joel Tenenbaum,Shlomo Havlin,H. Eugene Stanley

DOI: https://doi.org/10.1038/srep00313

2012-02-16

Abstract:We analyze the dynamic properties of 10^7 words recorded in English, Spanish and Hebrew over the period 1800--2008 in order to gain insight into the coevolution of language and culture. We report language independent patterns useful as benchmarks for theoretical models of language evolution. A significantly decreasing (increasing) trend in the birth (death) rate of words indicates a recent shift in the selection laws governing word use. For new words, we observe a peak in the growth-rate fluctuations around 40 years after introduction, consistent with the typical entry time into standard dictionaries and the human generational timescale. Pronounced changes in the dynamics of language during periods of war shows that word correlations, occurring across time and between words, are largely influenced by coevolutionary social, technological, and political factors. We quantify cultural memory by analyzing the long-term correlations in the use of individual words using detrended fluctuation analysis.

Physics and Society,Computation and Language,Information Retrieval,Adaptation and Self-Organizing Systems,Popular Physics
Application-Oriented Comparison and Evaluation of Six Semantic Similarity Measures Based on Wordnet

Peng-Yuan Liu,Tie-Jun Zhao,Xiao-Feng Yu

DOI: https://doi.org/10.1109/icmlc.2006.258857

2006-01-01

Abstract:In the task of auto-building a Chinese-English semantic lexicon for translation selection, this research presents a method, which introduces WordNet similarity measures to wash out misaligned Chinese-English word pairs. Six different proposed measures of similarity based on WordNet were experimentally compared and evaluated by using WordNet and the software package WordNet::Similarity. It was found that the leader measure is res and lch, then wup, lin takes the fourth place, then jcn, and random is the tailender
Understanding the Domain Development Through a Word Status Observation Model.

Tongyang Zhang,Ran Sun,Julia Fensel,Andrew Yu,Yi Bu,Jian Xu

DOI: https://doi.org/10.1016/j.joi.2023.101395

IF: 3.7

2023-01-01

Journal of Informetrics

Abstract:For the accurate scientific evaluation and advancement of science, it is crucial to understand the development state of a given disciplinary domain. Existing comprehension methodologies concentrate on quantitatively analyzing broad subject trends without considering the underlying complex status attributes of the words that support and enrich these surface-level trends. Through the perspective of the word role, this study deepens the examination of domain development in a more granular way. We use Word2vec to identify the representative semantic neighbors of a domain-specific feature word from literature. Then, a word status observation model is provided that classifies the role of these representative words as tree structures, including young leaves, dead leaves, roots, and trunk-branches, based on changes in their similarity ranking divergence and information entropy. The static word role provides a new insight into detecting the inter -nal detailed organizational composition and maturing status, while the dynamic role shift helps track historical development changes in the domain. Taking attention in psychology as a case by extracting psychology articles from Microsoft Academic Graph, the empirical study illustrates the results of the observation model and yields intriguing findings. First, the static word role po-sitioning shows that a word's status can be obviously different. Different status indicates areas currently keeping the states of thriving, obsolete, becoming firm supportive pillars (precipitating), and irregularly developing within the domain, respectively. Second, during different development periods, most of the words are maintaining the role of irregularly changing or tending towards precipitation. A certain proportion of words are tending towards prosperity, while only a small set of words are deviating from the prosperity state. The growth of attention-related studies as a whole is well-grown and gradually approaching maturity. Our study further supports researchers' understanding of the domain development status from a more granular perspective of word roles.
Chinese Evolution in Recent 150 Years: A Diachronic Study of Word Frequency in the Gospel of Mark

Cong Zhang,Haitao Liu

DOI: https://doi.org/10.1353/jcl.2017.0064

2019-01-01

Journal of Chinese Linguistics

Abstract:This study quantitatively investigated word frequencies in parallel Chinese texts at three different periods, namely, Classical Chinese (1855), Beijing Mandarin (1908) and Modern Mandarin (2010), attempting to shed some light on the evolution of the Chinese language over the last 150 years. With the evolution of Chinese: both the types and tokens increased diachronically, and sharp changes can be found in the high-frequency function words, vocabulary richness, and the distribution of word classes. The syntactic structure has also become increasingly complex. In addition, although the length of words (measured in character) has increased diachronically, disyllabic words have not always been preponderant in Chinese after the opium war of 1840–1842. The results show that the complication is a crucial trend in the evolution process of Chinese.
Co-word analysis and bibliometric visualization of translation quality literature: research topics and trends in the Chinese mainland (1997–2016)

Zhanhao Jiang,Yuan Tao

DOI: https://doi.org/10.1080/23306343.2018.1528750

2018-09-02

Asia Pacific Translation and Intercultural Studies

Abstract:Recent years has seen a steady development of translation quality research. This paper, with the help of Bicomb (for co-word analysis), UCINET(for social network analysis), and SPSS21.0 (for cluster analysis), has made a detailed exploration of papers on translation quality in CNKI (China National Knowledge Infrastructure) in terms of the number of papers, journal distribution, high-frequency key words, and diachronic change of key words that may help to reflect research topics and trends in the Chinese mainland in the past two decades. It also offers recommendations for further study and provides some insights for researchers on translation quality.
The Distribution of Synonymous Variants in Wenzhounese

Jieqiang Zhu,Haitao Liu

2018-01-01

Glottometrics

Abstract:This study investigates the diversification phenomenon of synonymy in Wenzhounese, a variety of Chinese spoken around the city of Wenzhou in southern China. Five groups of synonymous variants were extracted from Wenzhou Spoken Corpus (WSC) for finding the regularity of their rank-frequency distribution. The result revealed that the rank-frequency distribution of the investigated synonymous variants abides by the right-truncated modified Zipf-Alekseev distribution, a unified function modelling the diversifying process.
Visual exploration of software evolution via topic modeling

Huan Liu,Yubo Tao,Yining Qiu,Wenda Huang,Hai Lin

DOI: https://doi.org/10.1007/s12650-020-00739-7

IF: 1.7

2021-01-01

Journal of Visualization

Abstract:For various reasons, such as new requirements, architecture refactoring, and bug fixing, software projects often evolve to yield better quality and performance. All changes produced during the development process are reflected in the source code, which provides an opportunity to explore software evolution. In this paper, we propose a visual analytics system to support evolution analysis based on topic modeling. We focus on three aspects: (1) when significant changes to source code occur, (2) how software features evolve, and (3) why software evolution occurs. Each source file is regarded as a document and represented by its topic vector. The files of each two successive versions are classified into four types to quantify version differences, and the number of topic-associated files is denoted as the topic assignment to characterize feature evolution. Finally, we inspect the causes of software evolution through the visual comparison between versions. Two case studies on JavaScript libraries demonstrate the usefulness and effectiveness of our system.
The Statistical Research on Diachronic Changes of the Common Wordlist for Chinese Teaching

WANG Zhi-min

DOI: https://doi.org/10.3969/j.issn.1674-8174.2010.04.007

2010-01-01

Abstract:Frequency and time span of corpus are used to establish two statistical databases for common nouns,which can reflect the diachronic changes of Chinese nouns by designing different time points.The databases not only provide the valuable data for collecting the new words and filtering the historical words for Syllabus of Graded Words and Characters for Chinese Proficiency,but also provide the beneficial reference for artificial selection of the teaching vocabulary.In order to verify its reliabilty,we make a contrast analysis of the statistical database and high frequency words of textbook.The results show that the People's Daily,as a news corpus,contains the basic common nouns for Chinese as a second language teaching materials.
Chinese Word Co-occurrence Network: Its Small World Effect and Scale-free Property

Zhi-yuan LIU,Mao-song SUN

DOI: https://doi.org/10.3969/j.issn.1003-0077.2007.06.008

2007-01-01

Abstract:Some perspectives of human languages can be characterized by complex network analysis.In this paper,word co-occurrence networks for the Chinese language are automatically constructed based on very large manually word-segmented Chinese corpora with different size and style at first.Then systematic observations on these networks are made from the complex network's point of view.Experimental results show that these networks display two important features of complex networks:(1)The average distance between two words is 2.63-2.75,and the clustering coefficient is much greater than that given by a random network with the same parameters,which exhibits a typical small-world effect;and(2)The degree distributions of these networks generally obey the power-law,i.e.,the scale-free property.In addition,quantitative analysis is conducted for the kernel lexicons derived from these experiments.
Rank dynamics of word usage at multiple scales

José A. Morales,Ewan Colman,Sergio Sánchez,Fernanda Sánchez-Puig,Carlos Pineda,Gerardo Iñiguez,Germinal Cocho,Jorge Flores,Carlos Gershenson

DOI: https://doi.org/10.3389/fphy.2018.00045

2018-02-20

Abstract:The recent dramatic increase in online data availability has allowed researchers to explore human culture with unprecedented detail, such as the growth and diversification of language. In particular, it provides statistical tools to explore whether word use is similar across languages, and if so, whether these generic features appear at different scales of language structure. Here we use the Google Books $N$-grams dataset to analyze the temporal evolution of word usage in several languages. We apply measures proposed recently to study rank dynamics, such as the diversity of $N$-grams in a given rank, the probability that an $N$-gram changes rank between successive time intervals, the rank entropy, and the rank complexity. Using different methods, results show that there are generic properties for different languages at different scales, such as a core of words necessary to minimally understand a language. We also propose a null model to explore the relevance of linguistic structure across multiple scales, concluding that $N$-gram statistics cannot be reduced to word statistics. We expect our results to be useful in improving text prediction algorithms, as well as in shedding light on the large-scale features of language use, beyond linguistic and cultural differences across human populations.

Physics and Society
Evolution Path and Development Trend of the Research on N-V Shift

Yongjin Wu

DOI: https://doi.org/10.54097/nbka0305

2024-07-07

International Journal of Education and Humanities

Abstract:N-V shift is a widespread linguistic phenomenon in Chinese, and it is also an important part of the study of part of speech. Based on bibliometrics softwares, this paper takes the related papers of N-V shift in China as the research object, collects 426 related studies from CNKI in recent 30 years (1992-2023) as data sources, and explores the evolution path and trendency of N-V shift from five dimensions: author, stage publishing trend, keyword clustering and co-occurrence network, and theme time zone change, and presents its research status, frontier trend and prospect in theory and practice, ontology and application, in order to provide reference for the future development of this field.
Macroscopic and microscopic statistical properties observed in blog entries

Yukie Sano,Misako Takayasu

DOI: https://doi.org/10.48550/arXiv.0906.1744

2010-04-08

Abstract:We observe the statistical properties of blogs that are expected to reflect social human interaction. Firstly, we introduce a basic normalization preprocess that enables us to evaluate the genuine word frequency in blogs that are independent of external factors such as spam blogs, server-breakdowns, increase in the population of bloggers, and periodic weekly behaviors. After this process, we can confirm that small frequency words clearly follow an independent Poisson process as theoretically expected. Secondly, we focus on each blogger's basic behaviors. It is found that there are two kinds of behaviors of bloggers. Further, Zipf's law on word frequency is confirmed to be universally independent of individual activity types.

Physics and Society
A comparative study for wordnet guided text representation

Jian Zhang,Chunping Li

DOI: https://doi.org/10.1007/11589990_102

2005-01-01

Abstract:Text information processing depends critically on the proper text representation. A common and naïve way of representing a document is a bag of its component words [1], but the semantic relations between words are ignored, such as synonymy and hypernymy-hyponymy between nouns. This paper presents a model for representing a document in terms of the synonymy sets (synsets) in WordNet [2]. The synsets stand for concepts corresponding to the words of the document. The Vector Space Model describes a document as orthogonal term vectors. We replace terms with concepts to build Concept Vector Space Model (CVSM) for the training set. Our experiments on the Reuters Corpus Volume I (RCV1) dataset have shown that the result is satisfactory.

Statistical Overview Of Wordnet From 1.6 To 2.0

New Word Identification in Social Network Text Based on Time Series Information

The Natural Selection of Words: Finding the Features of Fitness

How Does Language Change As a Lexical Network? an Investigation Based on Written Chinese Word Co-Occurrence Networks

Wordnet Based Comparison Of Language Variation: A Study Based On Ccd And Cwn

Polish and English wordnets -- statistical analysis of interconnected networks

Empirical Analysis on a Keyword-Based Semantic System

The Global Organization of the English Lexicon and Its Evolution

Statistical Laws Governing Fluctuations in Word Use from Word Birth to Word Death

Application-Oriented Comparison and Evaluation of Six Semantic Similarity Measures Based on Wordnet

Understanding the Domain Development Through a Word Status Observation Model.

Chinese Evolution in Recent 150 Years: A Diachronic Study of Word Frequency in the Gospel of Mark

Co-word analysis and bibliometric visualization of translation quality literature: research topics and trends in the Chinese mainland (1997–2016)

The Distribution of Synonymous Variants in Wenzhounese

Visual exploration of software evolution via topic modeling

The Statistical Research on Diachronic Changes of the Common Wordlist for Chinese Teaching

Chinese Word Co-occurrence Network: Its Small World Effect and Scale-free Property

Rank dynamics of word usage at multiple scales

Evolution Path and Development Trend of the Research on N-V Shift

Macroscopic and microscopic statistical properties observed in blog entries

A comparative study for wordnet guided text representation