A Simple Derivation of the Heap's Law from the Generalized Zipf's Law

Leonid Boytsov

DOI: https://doi.org/10.48550/arXiv.1711.03066

2017-11-08

Information Retrieval

Abstract:I reproduce a rather simple formal derivation of the Heaps' law from the generalized Zipf's law, which I previously published in Russian.

What problem does this paper attempt to address?

Zipf'S Law Leads to Heaps' Law: Analyzing Their Relation in Finite-Size Systems

Linyuan Lue,Zi-Ke Zhang,Tao Zhou

DOI: https://doi.org/10.1371/journal.pone.0014139

IF: 3.7

2010-01-01

PLoS ONE

Abstract:Background: Zipf's law and Heaps' law are observed in disparate complex systems. Of particular interests, these two laws often appear together. Many theoretical models and analyses are performed to understand their co-occurrence in real systems, but it still lacks a clear picture about their relation.Methodology/Principal Findings: We show that the Heaps' law can be considered as a derivative phenomenon if the system obeys the Zipf's law. Furthermore, we refine the known approximate solution of the Heaps' exponent provided the Zipf's exponent. We show that the approximate solution is indeed an asymptotic solution for infinite systems, while in the finite-size system the Heaps' exponent is sensitive to the system size. Extensive empirical analysis on tens of disparate systems demonstrates that our refined results can better capture the relation between the Zipf's and Heaps' exponents.Conclusions/Significance: The present analysis provides a clear picture about the relation between the Zipf's law and Heaps' law without the help of any specific stochastic model, namely the Heaps' law is indeed a derivative phenomenon from the Zipf's law. The presented numerical method gives considerably better estimation of the Heaps' exponent given the Zipf's exponent and the system size. Our analysis provides some insights and implications of real complex systems. For example, one can naturally obtained a better explanation of the accelerated growth of scale-free networks.
Proper Interpretation of Heaps' and Zipf's Laws

Kim Chol-jun

2024-05-26

Abstract:We checked that the distribution of words in text should uniform, which gives Heaps' law as natural result, that is, the number of types of words can be expressed as a power law of the number of tokens within text. We developed a ``superposition'' model, which leads to an asymptotic power-law distribution of the number of occurrences (or frequency) of words, that is, Zipf's law. The model is well consistent with observations.

Physics and Society
Corrections of Zipf's and Heaps' Laws Derived from Hapax Rate Models

Łukasz Dębowski

2024-01-29

Abstract:The article introduces corrections to Zipf's and Heaps' laws based on systematic models of the proportion of hapaxes, i.e., words that occur once. The derivation rests on two assumptions: The first one is the standard urn model which predicts that marginal frequency distributions for shorter texts look as if word tokens were sampled blindly from a given longer text. The second assumption posits that the hapax rate is a simple function of the text length. Four such functions are discussed: the constant model, the Davis model, the linear model, and the logistic model. It is shown that the logistic model yields the best fit.

Computation and Language,Applications
A scaling law beyond Zipf's law and its relation to Heaps' law

Francesc Font-Clos,Gemma Boleda,Álvaro Corral

DOI: https://doi.org/10.1088/1367-2630/15/9/093033

2013-09-09

Abstract:The dependence with text length of the statistical properties of word occurrences has long been considered a severe limitation quantitative linguistics. We propose a simple scaling form for the distribution of absolute word frequencies which uncovers the robustness of this distribution as text grows. In this way, the shape of the distribution is always the same and it is only a scale parameter which increases linearly with text length. By analyzing very long novels we show that this behavior holds both for raw, unlemmatized texts and for lemmatized texts. For the latter case, the word-frequency distribution is well fit by a double power law, maintaining the Zipf's exponent value $\gamma\simeq 2$ for large frequencies but yielding a smaller exponent in the low frequency regime. The growth of the distribution with text length allows us to estimate the size of the vocabulary at each step and to propose an alternative to Heaps' law, which turns out to be intimately connected to Zipf's law, thanks to the scaling behavior.

Physics and Society,Statistical Mechanics,Adaptation and Self-Organizing Systems
Explanation and exact formula of Zipfs law evaluated from rank-share combinatorics

A Shyklo

DOI: https://doi.org/10.48550/arXiv.1705.07890

2017-05-20

Abstract:This work proves that ranks and shares are statistically dependent on one another, based on simple combinatorics. It presents a formula for rank-share distribution and illustrates that Zipfs law, is descended from expected values of various ranks in the new distribution. All conclusions, formulas and charts presented here were tested against publicly available statistical data in different areas. The correlation coefficient between the calculated values and statistical numbers provided by Bureau of Labor Statistics was 0.99899. Monte-Carlo simulations were performed as additional evidence.

Other Statistics,Physics and Society
Universality of Zipf's Law

Kenji Kawamura,Naomichi Hatano

DOI: https://doi.org/10.1143/JPSJ.71.1211

2002-03-29

Abstract:We introduce a simple and generic model that reproduces Zipf's law. By regarding the time evolution of the model as a random walk in the logarithmic scale, we explain theoretically why this model reproduces Zipf's law. The explanation shows that the behavior of the model is very robust and universal.

Statistical Mechanics,Disordered Systems and Neural Networks
Big Data and Large Numbers. Interpreting Zipf's Law

Horia-Nicolai L. Teodorescu

2023-05-08

Abstract:It turns out that some empirical facts in Big Data are the effects of properties of large numbers. Zipf's law 'noise' is an example of such an artefact. We expose several properties of the power law distributions and of similar distribution that occur when the population is finite and the rank and counts of elements in the population are natural numbers. We are particularly concerned with the low-rank end of the graph of the law, the potential of noise in the law, and with the approximation of the number of types of objects at various ranks. Approximations instead of exact solutions are the center of attention. Consequences in the interpretation of Zipf's law are discussed.

Physics and Society,Computation and Language,Statistics Theory
Deviation of Zipf's and Heaps' Laws in Human Languages with Limited Dictionary Sizes

Linyuan Lü,Zi-Ke Zhang,Tao Zhou

DOI: https://doi.org/10.1038/srep01082

IF: 4.6

2013-01-01

Scientific Reports

Abstract:Zipf's law on word frequency and Heaps' law on the growth of distinct words are observed in Indo-European language family, but it does not hold for languages like Chinese, Japanese and Korean. These languages consist of characters and are of very limited dictionary sizes. Extensive experiments show that: (i) The character frequency distribution follows a power law with exponent close to one, at which the corresponding Zipf's exponent diverges. Indeed, the character frequency decays exponentially in the Zipf's plot. (ii) The number of distinct characters grows with the text length in three stages: It grows linearly in the beginning, then turns to a logarithmical form and eventually saturates. A theoretical model for writing process is proposed, which embodies the rich-get-richer mechanism and the effects of limited dictionary size. Experiments, simulations and analytical solutions agree well with each other. This work refines the understanding about Zipf's and Heaps' laws in human language systems.
The Rank-Size Scaling Law and Entropy-Maximizing Principle

Yanguang Chen

DOI: https://doi.org/10.1016/j.physa.2011.07.010

2011-04-29

Abstract:The rank-size regularity known as Zipf's law is one of scaling laws and frequently observed within the natural living world and in social institutions. Many scientists tried to derive the rank-size scaling relation by entropy-maximizing methods, but the problem failed to be resolved thoroughly. By introducing a pivotal constraint condition, I present here a set of new derivations based on the self-similar hierarchy of cities. First, I derive a pair of exponent laws by postulating local entropy maximizing. From the two exponential laws follows a general hierarchical scaling law, which implies general Zipf's law. Second, I derive a special hierarchical scaling law with exponent equal to 1 by postulating global entropy maximizing, and this implies the strong form of Zipf's law. The rank-size scaling law proved to be one of the special cases of the hierarchical law, and the derivation suggests a certain scaling range with the first or last data point as an outlier. The entropy maximization of social systems differs from the notion of entropy increase in thermodynamics. For urban systems, entropy maximizing suggests the best equilibrium state of equity for parts/individuals and efficiency for the whole.

Physics and Society
The Mathematical Relationship Between Zipf'S Law And The Hierarchical Scaling Law

Yanguang Chen

DOI: https://doi.org/10.1016/j.physa.2011.12.031

2012-01-01

Abstract:The empirical studies of city-size distribution show that Zipfs law and the hierarchical scaling law are linked in many ways. The rank-size scaling and hierarchical scaling seem to be two different sides of the same coin, but their relationship has never been revealed by strict mathematical proof. In this paper, the Zipf's distribution of cities is abstracted as a q-sequence. Based on this sequence, a self-similar hierarchy consisting of many levels is defined and the numbers of cities in different levels form a geometric sequence. An exponential distribution of the average size of cities is derived from the hierarchy. Thus we have two exponential functions, from which follows a hierarchical scaling equation. The results can be statistically verified by simple mathematical experiments and observational data of cities. A theoretical foundation is then laid for the conversion from Zipf's law to the hierarchical scaling law, and the latter can show more information about city development than the former. Moreover, the self-similar hierarchy provides a new perspective for studying networks of cities as complex systems. A series of mathematical rules applied to cities such as the allometric growth law, the 2(n) principle and Pareto's law can be associated with one another by the hierarchical organization. (C) 2011 Elsevier B.V. All rights reserved.
Log-log Convexity of Type-Token Growth in Zipf's Systems

Francesc Font-Clos,Alvaro Corral

DOI: https://doi.org/10.1103/PhysRevLett.114.238701

2014-12-15

Abstract:It is traditionally assumed that Zipf's law implies the power-law growth of the number of different elements with the total number of elements in a system - the so-called Heaps' law. We show that a careful definition of Zipf's law leads to the violation of Heaps' law in random systems, and obtain alternative growth curves. These curves fulfill universal data collapses that only depend on the value of the Zipf's exponent. We observe that real books behave very much in the same way as random systems, despite the presence of burstiness in word occurrence. We advance an explanation for this unexpected correspondence.

Physics and Society,Data Analysis, Statistics and Probability
Theory of Zipf's Law and of General Power Law Distributions with Gibrat's law of Proportional Growth

A. Saichev,Y. Malevergne,D. Sornette

DOI: https://doi.org/10.48550/arXiv.0808.1828

2008-08-13

Abstract:We summarize a book under publication with his title written by the three present authors, on the theory of Zipf's law, and more generally of power laws, driven by the mechanism of proportional growth. The preprint is available upon request from the authors. For clarity, consistence of language and conciseness, we discuss the origin and conditions of the validity of Zipf's law using the terminology of firms' asset values. We use firms at the entities whose size distributions are to be explained. It should be noted, however, that most of the relations discussed in this book, especially the intimate connection between Zipf's and Gilbrat's laws, underlie Zipf's law in diverse scientific areas. The same models and variations thereof can be straightforwardly applied to any of the other domains of application.

General Finance,Data Analysis, Statistics and Probability,Physics and Society
Zipf's Law for Atlas Models

Ricardo T. Fernholz,Robert Fernholz

DOI: https://doi.org/10.48550/arXiv.1707.04285

2020-06-03

Abstract:A set of data with positive values follows a Pareto distribution if the log-log plot of value versus rank is approximately a straight line. A Pareto distribution satisfies Zipf's law if the log-log plot has a slope of -1. Since many types of ranked data follow Zipf's law, it is considered a form of universality. We propose a mathematical explanation for this phenomenon based on Atlas models and first-order models, systems of positive continuous semimartingales with parameters that depend only on rank. We show that the stable distribution of an Atlas model will follow Zipf's law if and only if two natural conditions, conservation and completeness, are satisfied. Since Atlas models and first-order models can be constructed to approximate systems of time-dependent rank-based data, our results can explain the universality of Zipf's law for such systems. However, ranked data generated by other means may follow non-Zipfian Pareto distributions. Hence, our results explain why Zipf's law holds for word frequency, firm size, household wealth, and city size, while it does not hold for earthquake magnitude, cumulative book sales, the intensity of solar flares, and the intensity of wars, all of which follow non-Zipfian Pareto distributions.

General Economics,Physics and Society
A Rank-Based Approach to Zipf's Law

Ricardo T. Fernholz,Robert Fernholz

DOI: https://doi.org/10.48550/arXiv.1602.08533

2016-02-27

Abstract:An Atlas model is a rank-based system of continuous semimartingales for which the steady-state values of the processes follow a power law, or Pareto distribution. For a power law, the log-log plot of these steady-state values versus rank is a straight line. Zipf's law is a power law for which the slope of this line is -1. In this note, rank-based conditions are found under which an Atlas model will follow Zipf's law. An advantage of this rank-based approach is that it provides information about the dynamics of systems that result in Zipf's law.

General Economics,Mathematical Finance
Zipf's Law for Cities: An Explanation

Xavier Gabaix,X. Gabaix

DOI: https://doi.org/10.1162/003355399556133

IF: 19.013

1999-08-01

The Quarterly Journal of Economics

Abstract:Zipf's law is a very tight constraint on the class of admissible models of local growth. It says that for most countries the size distribution of cities strikingly fits a power law: the number of cities with populations greater than S is proportional to 1/S. Suppose that, at least in the upper tail, all cities follow some proportional growth process (this appears to be verified emperically). This automatically leads their distribution to converge to Zipf's law.

economics
Maximal nonsymmetric entropy leads naturally to Zipf's law

Chengshi Liu

DOI: https://doi.org/10.48550/arXiv.cond-mat/0609344

2006-09-14

Disordered Systems and Neural Networks

Abstract:As the most fundamental empirical law, Zipf's law has been studied from many aspects. But its meaning is still an open problem. Some models have been constructed to explain Zipf's law. In the letter, a new concept named nonsymmetric entropy was introduced, maximizing nonsymmetric entropy leads naturally to Zipf's law.
Integers and Prime Numbers: Deriving Zipf's Law

Helmut Satz

2024-03-13

Abstract:In a prime number decomposition of integers in a given set, the occurrence frequencies of prime numbers are shown to satisfy a general forms of Zipf's law.

Physics and Society,Number Theory
Maximal Diversity and Zipf's Law

Onofrio Mazzarisi,Amanda de Azevedo-Lopes,Jeferson J. Arenzon,Federico Corberi

DOI: https://doi.org/10.1103/PhysRevLett.127.128301

2021-10-05

Abstract:Zipf's law describes the empirical size distribution of the components of many systems in natural and social sciences and humanities. We show, by solving a statistical model, that Zipf's law co-occurs with the maximization of the diversity of the component sizes. The law ruling the increase of such diversity with the total dimension of the system is derived and its relation with Heaps' law is discussed. As an example, we show that our analytical results compare very well with linguistics datasets.

Statistical Mechanics
The common patterns of abundance: the log series and Zipf's law

Steven A. Frank

DOI: https://doi.org/10.12688/f1000research.18681.1

2019-01-12

Abstract:In a language corpus, the probability that a word occurs $n$ times is often proportional to $1/n^2$. Assigning rank, $s$, to words according to their abundance, $\log s$ vs $\log n$ typically has a slope of minus one. That simple Zipf's law pattern also arises in the population sizes of cities, the sizes of corporations, and other patterns of abundance. By contrast, for the abundances of different biological species, the probability of a population of size $n$ is typically proportional to $1/n$, declining exponentially for larger $n$, the log series pattern. This article shows that the differing patterns of Zipf's law and the log series arise as the opposing endpoints of a more general theory. The general theory follows from the generic form of all probability patterns as a consequence of conserved average values and the associated invariances of scale. To understand the common patterns of abundance, the generic form of probability distributions plus the conserved average abundance is sufficient. The general theory includes cases that are between the Zipf and log series endpoints, providing a broad framework for analyzing widely observed abundance patterns.

Populations and Evolution,Statistical Mechanics,Probability
A universal rank-size law

Marcel Ausloos,Roy Cerqueti

DOI: https://doi.org/10.1371/journal.pone.0166011

2016-11-05

Abstract:A mere hyperbolic law, like the Zipf's law power function, is often inadequate to describe rank-size relationships. An alternative theoretical distribution is proposed based on theoretical physics arguments starting from the Yule-Simon distribution. A modeling is proposed leading to a universal form. A theoretical suggestion for the "best (or optimal) distribution", is provided through an entropy argument. The ranking of areas through the number of cities in various countries and some sport competition ranking serves for the present illustrations.

Data Analysis, Statistics and Probability,Physics and Society

A Simple Derivation of the Heap's Law from the Generalized Zipf's Law

Zipf'S Law Leads to Heaps' Law: Analyzing Their Relation in Finite-Size Systems

Proper Interpretation of Heaps' and Zipf's Laws

Corrections of Zipf's and Heaps' Laws Derived from Hapax Rate Models

A scaling law beyond Zipf's law and its relation to Heaps' law

Explanation and exact formula of Zipfs law evaluated from rank-share combinatorics

Universality of Zipf's Law

Big Data and Large Numbers. Interpreting Zipf's Law

Deviation of Zipf's and Heaps' Laws in Human Languages with Limited Dictionary Sizes

The Rank-Size Scaling Law and Entropy-Maximizing Principle

The Mathematical Relationship Between Zipf'S Law And The Hierarchical Scaling Law

Log-log Convexity of Type-Token Growth in Zipf's Systems

Theory of Zipf's Law and of General Power Law Distributions with Gibrat's law of Proportional Growth

Zipf's Law for Atlas Models

A Rank-Based Approach to Zipf's Law

Zipf's Law for Cities: An Explanation

Maximal nonsymmetric entropy leads naturally to Zipf's law

Integers and Prime Numbers: Deriving Zipf's Law

Maximal Diversity and Zipf's Law

The common patterns of abundance: the log series and Zipf's law

A universal rank-size law