Conditional Equi-concentration of Types

Marian Grendar
2007-01-12
Abstract: Conditional Equi-concentration of Types on I-projections is presented. It provides an extension of Conditional Weak Law of Large Numbers to the case of several I-projections. Also a multiple I-projections extension of Gibbs Conditioning Principle is developed. mu-projection variants of the probabilistic laws are stated. Implications of the results for Relative Entropy Maximization, Maximum Probability, Maximum Entropy in the Mean and Maximum Renyi-Tsallis Entropy methods are discussed.
Probability
What problem does this paper attempt to address?
The main problem this paper attempts to address is the extension of the Conditional Weak Law of Large Numbers (CWLLN) and the Gibbs Conditioning Principle (GCP) to handle the case of multiple I-projections. Specifically: 1. **Conditional Equi-concentration of Types (ICET)**: When the feasible set \(\Pi\) contains multiple I-projections, the paper explores the phenomenon of type conditional equi-concentration on these I-projections. ICET indicates that as the sample size \(n\) increases, the types conditionally concentrate equi-probably on each appropriate I-projection. 2. **Extended Gibbs Conditioning Principle (EGCP)**: The paper extends the GCP to the case of multiple I-projections, indicating that the conditional probability can be represented as an equal-weight mixture of the probabilities of these I-projections. Additionally, the paper introduces the concept of \(\mu\)-projection and proposes the \(\mu\)-projection versions of the Conditional Weak Law of Large Numbers (\(\mu\)-CWLLN), Gibbs Conditioning Principle (\(\mu\)-GCP), Conditional Equi-concentration of Types (\(\mu\)-CET), and the Extended Gibbs Conditioning Principle (Boltzmann Conditioning Principle, BCP). These results are significant for understanding the relationship between the Maximum Probability Method (MaxProb) and the Maximum Entropy Method (MaxEnt). ### Main Contributions 1. **ICET and EGCP**: Extended the existing Conditional Weak Law of Large Numbers and Gibbs Conditioning Principle to apply to the case of multiple I-projections. 2. **\(\mu\)-projection**: Introduced the concept of \(\mu\)-projection and demonstrated its asymptotic consistency with I-projection. 3. **Method Comparison**: Discussed the application and interrelation of the Maximum Probability Method (MaxProb) and the Maximum Entropy Method (MaxEnt) in the Boltzmann Jaynes inverse problem. 4. **Impact on Other Methods**: Explored the issues with the Maximum Rényi-Tsallis Entropy Method (maxTent) in selecting probability distributions, pointing out that it may choose distributions whose conditional probabilities asymptotically approach zero. ### Application Background These theoretical results have important applications in fields such as statistical physics, information theory, and machine learning. For example, the phenomenon of conditional equi-concentration of types is similar to the phase coexistence phenomenon in thermodynamics (such as the triple point of water, steam, and ice), while the Maximum Probability Method and Maximum Entropy Method provide different perspectives and tools for handling inverse problems.