Range-Renewal Speed and Entropy for I.I.D Models

xinxing chen,jiansheng xie,jiangang ying

2013-01-01

Abstract: In this note the relation between the range-renewal speed and entropy for i.i.d. models is discussed.

What problem does this paper attempt to address?

Ruin probabilities of a two-dimensional risk model with dependent risks of heavy tail

xinmei shen,yi zhang

DOI: https://doi.org/10.1016/j.spl.2013.03.029

IF: 0.718

2013-01-01

Statistics & Probability Letters

Abstract:This paper considers a two-dimensional discrete time risk model with constant interest rates, and individual net losses in ERV(−α,−β), the class of extended regular variations with indices 0<α≤β<∞. Some asymptotic results for both finite-time and infinite-time ruin probabilities under two types of ruin times are established. The two components of net losses are allowed to be generally dependent.
Risk Model for Interest Randomness

张奕,李志英

DOI: https://doi.org/10.3321/j.issn:1008-9497.2004.02.004

2004-01-01

Abstract:Risk model for interest randomness is considered. The moments of claim size are caculated under the force of interest accumulation function as a statinary and independent increment process. If the interest randomness is Wiener process and the loss of distribution is pareto distribution, then the expression of moments of the claim size is more concrete and practical.
Bidimensional Discrete-Time Risk Model with Constant Interest

WANG Quan,ZHANG Yi

DOI: https://doi.org/10.3785/j.issn.1008-9497.2008.05.006

2008-01-01

Abstract:Including constant interest effect, a bidimensional discrete-time risk model is concerned. Using the martingale technique, the infinite-time ruin probability a Lundberg-type inequality and upper bound are obtained. Then some numerical examples are given in order to illustrate the effect of different parameters.
Range-Renewal Structure of I.I.D. Samplings

lihong wang,jiansheng xie

2013-01-01

Abstract: In this note the range-renewal structure of general independent and identically distributed samplings is discussed, which is a natural extension of the result in Chen et al (arXiv:1305.1829).
The ruin probability of the renewal model with constant interest force and upper-tailed independent heavy-tailed claims

Xin Mei Shen,Zheng Yan Lin

DOI: https://doi.org/10.1007/s10114-010-7574-0

2010-01-01

Abstract:In this paper, we obtain the uniform estimate for discounted aggregate claims in the continuous-time renewal model of upper-tailed independent and heavy-tailed random variables. With constant interest force and constant premium rate, we establish a uniform simple asymptotic formula for ruin probability of the renewal model in the case where the initial surplus is large.
An Information Theoretic Interpretation for H-Infinity Entropy

H Zhang,YX Sun

DOI: https://doi.org/10.1109/cacsd.2002.1036971

2002-01-01

Abstract:The relation between mutual information rate of system variables and system H-infinity entropy is formulated, an information theoretic interpretation of minimum entropy H-infinity control method is discussed.
CLT, MDP and LDP for Range-Renewals of I.I.D.Samplings from an Infinite Discrete Distribution

Xin-Xing Chen,Jian-Sheng Xie,Min-Zhi Zhao

DOI: https://doi.org/10.48550/arxiv.2211.08815

2022-01-01

Abstract: Let $R_n$ be the number of distinct values of the $n$ simple samples from an infinite discrete distribution. In 1960 Bahadure proved $\displaystyle \lim_{n\to \infty} \frac{R_n}{\Enum R_n}=1$ in probability; Chen et al. proved the limit in the sense of almost sure convergence, along with other results. In this note we present results of CLT, MDP and LDP for $R_n$ under mild conditions.
Range-Renewal Structure of Transient Simple Random Walk

Jian-Sheng Xie,Yuan-Yuan Xu

DOI: https://doi.org/10.1016/j.spl.2013.05.031

IF: 0.718

2013-01-01

Statistics & Probability Letters

Abstract:Dvoretzky and Erdös (1950) proved limn→+∞Rnn=γ almost surely for the simple symmetric random walk on Zd (d≥3) with Rn being the number of distinct cites visited in the time interval [0,n) and γ=γd being the escape rate. Following Derriennic’s approach, we prove further in this note that limn→+∞Rn,kRn=γ(1−γ)k−1,limn→+∞Rn,kRn,k+=γ,k=1,2,⋯ almost surely with Rn,k (resp. Rn,k+) being the number of distinct cites visited exactly (resp. at least) k times in the time interval [0,n). Such results are generalized to transient simple random walks on discrete groups.
Properties of the Shannon, Rényi and other entropies: dependence in parameters, robustness in distributions and extremes

Iryna Bodnarchuk,Yuliya Mishura,Kostiantyn Ralchenko

2024-11-24

Abstract:We calculate and analyze various entropy measures and their properties for selected probability distributions. The entropies considered include Shannon, Rényi, generalized Rényi, Tsallis, Sharma-Mittal, and modified Shannon entropy, along with the Kullback-Leibler divergence. These measures are examined for several distributions, including gamma, chi-squared, exponential, Laplace, and log-normal distributions. We investigate the dependence of the entropy on the parameters of the respective distribution. We also study the convergence of Shannon entropy for certain probability distributions. Furthermore, we identify the extreme values of Shannon entropy for Gaussian vectors.

Information Theory,Probability
Universal R\'enyi Entropy of Quasiparticle Excitations

Jiaju Zhang,M. A. Rajabpour

2020-01-01

Abstract:The Renyi entropies of quasiparticle excitations in the many-body gapped systems show a remarkable universal picture which can be understood partially by combination of a semiclassical argument with the quantum effect of (in)distinguishability. The universal Renyi entropies are independent of the model, the quasiparticle momenta, and the connectedness of the subsystem. In this letter we calculate exactly the single-interval and double-interval Renyi entropies of quasiparticle excitations in the many-body gapped fermions, bosons, and XY chains. We find additional contributions to the universal Renyi entropy in the excited states with quasiparticles of different momenta. The additional terms are different in the fermionic and bosonic chains, depend on the momentum differences of the quasiparticles, and are different for the single interval and the double interval. We derive the analytical Renyi entropy in the extremely gapped limit, matching perfectly the numerical results as long as either the intrinsic correlation length of the model or all the de Broglie wavelengths of the quasiparticles are small. When the momentum difference of any pair of distinct quasiparticles is small, the additional terms are non-negligible. On the contrary, when the difference of the momenta of each pair of distinct quasiparticles is large, the additional terms could be neglected. The universal single-interval Renyi entropy and its additional terms in the XY chain are the same as those in the fermionic chain, while the universal Renyi entropy of the double intervals and its additional terms are different, due to the fact that the local degrees of freedom of the XY chain are the Pauli matrices not the spinless fermions. We argue that the derived formulas have universal properties and can be applied for a wider range of models than those discussed here.
Generalized Relative Entropy: New Look at Rényi Entropy and Its Exploration from Complexity Measures to Sparsity Measures with Applications in Machine Condition Monitoring

Lan Wei,Dong Wang,Yu Wang

DOI: https://doi.org/10.1016/j.ymssp.2024.111917

IF: 8.4

2025-01-01

Mechanical Systems and Signal Processing

Abstract:Complexity measures and sparsity measures are capable of detecting and characterizing underlying dynamic changes in machine condition monitoring. Previous studies have developed Shannon entropy and Kullback-Leibler (KL) divergence and their generalization, namely Renyi entropy and Renyi divergence, but their theoretical properties as a function of sparsity and their transitions from complexity measures to sparsity measures are not fully explored and discussed. This paper continues conducting theoretical and experimental investigations on Renyi entropy and Renyi divergence and exploring their theoretical properties and transitions to enrich the domain of complexity and sparsity measures and their applications to machine condition monitoring. Specifically, this paper theoretically proves and verifies that Renyi entropy has a similar "bilateral reduction" effect exhibited by other complexity measures including correlation dimension, approximate entropy, fuzzy entropy and sample entropy. Inspired by relative entropy, which is a special case of KL divergence and a sparsity measure, and the relationship between KL divergence and Renyi divergence, in this paper, we attempt to generalize relative entropy by adding an order of alpha, to convert a series of complexity measures from Renyi entropy to a series of sparsity measures, which answers how to realize transitions from complexity measures to sparsity measures. More importantly, this paper is helpful to provide a guideline for correct uses of Renyi entropy, Renyi divergence, and generalized relative entropy for the quantification of impulsive transients caused by machine faults for machine condition monitoring.
Rényi entropy, Rényi divergence and Jensen-Rényi information generating functions, and associated properties and estimation

Shital Saha,Suchandan Kayal,N. Balakrishnan

2024-05-22

Abstract:In this paper, we propose Rényi information generating function (RIGF) and discuss its various properties. The relation between the RIGF and Shannon entropy of order $q>0$ is established. Several bounds are obtained. The RIGF of escort distribution is also derived. Furthermore, we introduce Rényi divergence information generating function (RDIGF) and discuss its effect under monotone transformations. Next, we propose Jensen-Rényi information generating function (JRIGF) and establish its properties. In addition, we present non-parametric and parametric estimators of the RIGF. For illustrative purpose, a simulation study is carried out and a real data relating to the failure times of electronic components is analyzed. Finally, a comparison study between the non-parametric and parametric estimators is made in terms of absolute bias and mean square error (MSE).

Statistics Theory
Entropy dimension for Deterministic Walks in Random Sceneries

Dou Dou,Kyewon Koh Park

DOI: https://doi.org/10.1017/etds.2021.41

2020-12-24

Abstract:Entropy dimension is an entropy-type quantity which takes values in $[0,1]$ and classifies different levels of intermediate growth rate of complexity for dynamical systems. In this paper, we consider the complexity of skew products of irrational rotations with Bernoulli systems, which can be viewed as deterministic walks in random sceneries, and show that this class of models can have any given entropy dimension by choosing suitable rotations for the base system.

Dynamical Systems
Assessment of Time Series Complexity Using Improved Approximate Entropy

Kong De-Ren,Xie Hong-Bo

DOI: https://doi.org/10.1088/0256-307x/28/9/090502

2011-01-01

Chinese Physics Letters

Abstract:Approximate entropy (ApEn), a measure quantifying complexity and/or regularity, is believed to be an effective method of analyzing diverse settings. However, the similarity definition of vectors based on Heaviside function may cause some problems in the validity and accuracy of ApEn. To overcome the problems, an improved approximate entropy (iApEn) based on the sigmoid function is proposed. The performance of iApEn is tested on the independent identically distributed (IID) Gaussian noise, the MIX stochastic model, the Rossler map, the logistic map, and the high-dimensional Mackey-Glass oscillator. The results show that iApEn is superior to ApEn in several aspects, including better relative consistency, freedom of parameter selection, robust to noise, and more independence on record length when characterizing time series with different complexities.
Rényi complexity in mean-field disordered systems

Nina Javerzat,Eric Bertin,Misaki Ozawa

2024-11-30

Abstract:Configurational entropy, or complexity, plays a critical role in characterizing disordered systems such as glasses, yet its measurement often requires significant computational resources. Recently, Rényi entropy, a one-parameter generalization of Shannon entropy, has gained attention across various fields of physics due to its simpler functional form, making it more practical for measurements. In this paper, we compute the Rényi version of complexity for prototypical mean-field disordered models, including the random energy model, its generalization, referred to as the random free energy model, and the $p$-spin spherical model. We first demonstrate that the Rényi complexity with index $m$ is related to the free energy difference for a generalized annealed Franz-Parisi potential with $m$ clones. Detailed calculations show that for models having one-step replica symmetry breaking (RSB), the Rényi complexity vanishes at the Kauzmann transition temperature $T_K$, irrespective of $m>1$, while RSB solutions are required even in the liquid phase. This study strengthens the link between Rényi entropy and the physics of disordered systems and provides theoretical insights for its practical measurements.

Disordered Systems and Neural Networks,Statistical Mechanics
SOME NEW RESULTS ON RÉNYI ENTROPY OF RESIDUAL LIFE AND INACTIVITY TIME

Xiaohu Li,Shuhong Zhang

DOI: https://doi.org/10.1017/s0269964810000379

2011-01-01

Probability in the Engineering and Informational Sciences

Abstract:This article deals with Rényi entropies for the residual life and the inactivity time. Monotonic properties of the entropy in order statistics, record values, and weighted distributions are investigated, and the comparison on weighted random variables is studied in terms of residual Rényi entropy as well.
Investigations on Collision Entropy and Its Variant for Machine Condition Monitoring

Lan Wei,Dong Wang,Yu Wang,Ming Yan

DOI: https://doi.org/10.1088/1742-6596/2853/1/012069

2024-01-01

Journal of Physics Conference Series

Abstract:Abstract The efficacy of sparsity measures and complexity measures in machine condition monitoring is well acknowledged. However, a comprehensive exploration of their theoretical underpinnings and interrelations remains incomplete. This study endeavors to address this gap by employing collision entropy as an illustrative case and establishing its relationship with sparsity measures. Through mathematical derivations, it is demonstrated that collision entropy exhibits a phenomenon termed as “bilateral reduction,” a characteristic indicative of complexity measures. Motivated by foundational concepts such as Shannon entropy and relative entropy, this paper introduces the concept of variation in collision entropy, which serves as a sparsity measure. Notably, both Shannon entropy and collision entropy, as specific cases of Rényi entropy, manifest the “bilateral reduction” effect and can be transformed into sparsity measures through certain methodologies. These insights contribute significantly to the exploration of the theoretical properties of Rényi entropy and its conversion into a cluster of sparse measures. This research augments the understanding of sparsity measures and complexity measures, offering theoretical insights crucial for their practical deployment in real-world applications.
Measuring Renyi Entropy in Neural Network Quantum States

Han-Qing Shi,Hai-Qing Zhang

2024-03-14

Abstract:We compute the Renyi entropy in a one-dimensional transverse-field quantum Ising model by employing a swapping operator acting on the states which are prepared from the neural network methods. In the static ground state, Renyi entropy can uncover the critical point of the quantum phase transition from paramagnetic to ferromagnetic. At the critical point, the relation between the Renyi entropy and the subsystem size satisfies the predictions from conformal field theory. In the dynamical case, we find coherent oscillations of the Renyi entropy after the end of the linear quench. These oscillations have universal frequencies which may come from the superpositions of excited states. The asymptotic form of the Renyi entropy implies a new length scale away from the critical point. This length scale is also verified by the overlap of the reduced Renyi entropy against the dimensionless subsystem size.

Disordered Systems and Neural Networks,High Energy Physics - Theory,Quantum Physics
Renewal model for dependent binary sequences

Marco Zamparo

DOI: https://doi.org/10.1007/s10955-022-02893-8

2021-08-25

Abstract:We suggest to construct infinite stochastic binary sequences by associating one of the two symbols of the sequence with the renewal times of an underlying renewal process. Focusing on stationary binary sequences corresponding to delayed renewal processes, we investigate correlations and the ability of the model to implement a prescribed autocovariance structure, showing that a large variety of subexponential decay of correlations can be accounted for. In particular, robustness and efficiency of the method are tested by generating binary sequences with polynomial and stretched-exponential decay of correlations. Moreover, to justify the maximum entropy principle for model selection, an asymptotic equipartition property for typical sequences that naturally leads to the Shannon entropy of the waiting time distribution is demonstrated. To support the comparison of the theory with data, a law of large numbers and a central limit theorem are established for the time average of general observables.

Mathematical Physics,Statistical Mechanics,Probability
Range-Renewal Processes: SLLN, Power Law and Beyonds

xinxing chen,jiansheng xie,jiangang ying

2013-01-01

Abstract: Given $n$ samples of a regular discrete distribution $\pi$, we prove in this article first a serial of SLLNs results (of Dvoretzky and Erd\"{o}s' type) which implies a typical power law when $\pi$ is heavy-tailed. Constructing a (random) graph from the ordered $n$ samples, we can establish other laws for the degree-distribution of the graph. The phenomena of small world is also discussed.

Range-Renewal Speed and Entropy for I.I.D Models

Ruin probabilities of a two-dimensional risk model with dependent risks of heavy tail

Risk Model for Interest Randomness

Bidimensional Discrete-Time Risk Model with Constant Interest

Range-Renewal Structure of I.I.D. Samplings

The ruin probability of the renewal model with constant interest force and upper-tailed independent heavy-tailed claims

An Information Theoretic Interpretation for H-Infinity Entropy

CLT, MDP and LDP for Range-Renewals of I.I.D.Samplings from an Infinite Discrete Distribution

Range-Renewal Structure of Transient Simple Random Walk

Properties of the Shannon, Rényi and other entropies: dependence in parameters, robustness in distributions and extremes

Universal R\'enyi Entropy of Quasiparticle Excitations

Generalized Relative Entropy: New Look at Rényi Entropy and Its Exploration from Complexity Measures to Sparsity Measures with Applications in Machine Condition Monitoring

Rényi entropy, Rényi divergence and Jensen-Rényi information generating functions, and associated properties and estimation

Entropy dimension for Deterministic Walks in Random Sceneries

Assessment of Time Series Complexity Using Improved Approximate Entropy

Rényi complexity in mean-field disordered systems

SOME NEW RESULTS ON RÉNYI ENTROPY OF RESIDUAL LIFE AND INACTIVITY TIME

Investigations on Collision Entropy and Its Variant for Machine Condition Monitoring

Measuring Renyi Entropy in Neural Network Quantum States

Renewal model for dependent binary sequences

Range-Renewal Processes: SLLN, Power Law and Beyonds