Entropy Rate for Hidden Markov Chains with rare transitions

Yuval Peres,Anthony Quas

DOI: https://doi.org/10.48550/arXiv.1012.2086

2010-12-09

Information Theory

Abstract:We consider Hidden Markov Chains obtained by passing a Markov Chain with rare transitions through a noisy memoryless channel. We obtain asymptotic estimates for the entropy of the resulting Hidden Markov Chain as the transition rate is reduced to zero.

What problem does this paper attempt to address?

Shannon Entropy Rate of Hidden Markov Processes

Alexandra M. Jurgens,James P. Crutchfield

DOI: https://doi.org/10.1007/s10955-021-02769-3

2021-05-01

Journal of Statistical Physics

Abstract:Abstract Hidden Markov chains are widely applied statistical models of stochastic processes, from fundamental physics and chemistry to finance, health, and artificial intelligence. The hidden Markov processes they generate are notoriously complicated, however, even if the chain is finite state: no finite expression for their Shannon entropy rate exists, as the set of their predictive features is generically infinite. As such, to date one cannot make general statements about how random they are nor how structured. Here, we address the first part of this challenge by showing how to efficiently and accurately calculate their entropy rates. We also show how this method gives the minimal set of infinite predictive features. A sequel addresses the challenge’s second part on structure.

physics, mathematical
Some Generalized Entropy Ergodic Theorems for Nonhomogeneous Hidden Markov Models

Qifeng Yao,Longsheng Cheng,Wenhe Chen,Ting Mao

DOI: https://doi.org/10.3390/math12040605

IF: 2.4

2024-02-18

Mathematics

Abstract:Entropy measures the randomness or uncertainty of a stochastic process, and the entropy rate refers to the limit of the time average of entropy. The generalized entropy rate in the form of delayed averages can overcome the redundancy of initial information while ensuring stationarity. Therefore, it has better practical value. A Hidden Markov Model (HMM) contains two stochastic processes, a stochastic process in which all states can be observed and a Markov chain in which all states cannot be observed. The entropy rate is an important characteristic of HMMs. The transition matrix of a homogeneous HMM is unique, while a Nonhomogeneous Hidden Markov Model (NHMM) requires the transition matrices to be dependent on time variables. From the perspective of model structure, NHMMs are novel extensions of homogeneous HMMs. In this paper, the concepts of the generalized entropy rate and NHMMs are defined and fully explained, a strong limit theorem and limit properties of a norm are presented, and then generalized entropy ergodic theorems with an almost surely convergence for NHMMs are obtained. These results provide concise formulas for the computation and estimation of the generalized entropy rate for NHMMs.

mathematics
Analyticity of Entropy Rate of Hidden Markov Chains

Guangyue Han,Brian Marcus

DOI: https://doi.org/10.48550/arXiv.math/0507235

2006-04-04

Abstract:We prove that under mild positivity assumptions the entropy rate of a hidden Markov chain varies analytically as a function of the underlying Markov chain parameters. A general principle to determine the domain of analyticity is stated. An example is given to estimate the radius of convergence for the entropy rate. We then show that the positivity assumptions can be relaxed, and examples are given for the relaxed conditions. We study a special class of hidden Markov chains in more detail: binary hidden Markov chains with an unambiguous symbol, and we give necessary and sufficient conditions for analyticity of the entropy rate for this case. Finally, we show that under the positivity assumptions the hidden Markov chain {\em itself} varies analytically, in a strong sense, as a function of the underlying Markov chain parameters.

Probability,Information Theory
An efficient algorithm for the entropy rate of a hidden Markov model with unambiguous symbols

Jaideep Mulherkar

DOI: https://doi.org/10.48550/arXiv.1402.2733

2014-02-18

Abstract:We demonstrate an efficient formula to compute the entropy rate $H(\mu)$ of a hidden Markov process with $q$ output symbols where at least one symbol is unambiguously received. Using an approximation to $H(\mu)$ to the first $N$ terms we give a $O(Nq^3$) algorithm to compute the entropy rate of the hidden Markov model. We use the algorithm to estimate the entropy rate when the parameters of the hidden Markov model are <a class="link-external link-http" href="http://unknown.In" rel="external noopener nofollow">this http URL</a> the case of $q =2$ the process is the output of the Z-channel and we use this fact to give bounds on the capacity of the Gilbert channel.

Information Theory
On the Entropy Rate of Hidden Markov Processes Observed Through Arbitrary Memoryless Channels

Jun Luo,Dongning Guo

DOI: https://doi.org/10.1109/tit.2009.2013030

IF: 2.5

2009-01-01

IEEE Transactions on Information Theory

Abstract:This paper studies the entropy and filtering of hidden Markov processes (HMPs) which are the observation of a binary homogeneous Markov chain through an arbitrary memoryless channel. A fixed-point functional equation is derived for the stationary distribution of an input symbol conditioned on all past observations. The entropy or differential entropy rate of the HMP can then be computed in two ways: one by exploiting the average entropy of each input symbol conditioned on past observations, and the other by applying a differential relationship between the input-output mutual information and the stationary distribution obtained via filtering. While the existence of a solution to the fixed-point equation is guaranteed by martingale theory, its uniqueness follows from the fact that the solution is the fixed point of a contraction mapping. Due to lack of an analytical solution to the fixed-point equation, a numerical method is proposed in which the fixed-point functional equation is first converted to a discrete linear system using uniform quantization and then solved using quadratic programming. Two examples, which the numerical method is applied to the binary symmetric channel (BSC) and additive white Gaussian noise (AWGN) channel, are presented. Unlike many other numerical methods, this numerical solution is not based on averaging over a long sample path of the HMP.
Entropy estimators for Markovian sequences: A comparative analysis

Juan De Gregorio,David Sanchez,Raul Toral

DOI: https://doi.org/10.3390/e26010079

2024-01-18

Abstract:Entropy estimation is a fundamental problem in information theory that has applications in various fields, including physics, biology, and computer science. Estimating the entropy of discrete sequences can be challenging due to limited data and the lack of unbiased estimators. Most existing entropy estimators are designed for sequences of independent events and their performance vary depending on the system being studied and the available data size. In this work we compare different entropy estimators and their performance when applied to Markovian sequences. Specifically, we analyze both binary Markovian sequences and Markovian systems in the undersampled regime. We calculate the bias, standard deviation and mean squared error for some of the most widely employed estimators. We discuss the limitations of entropy estimation as a function of the transition probabilities of the Markov processes and the sample size. Overall, this paper provides a comprehensive comparison of entropy estimators and their performance in estimating entropy for systems with memory, which can be useful for researchers and practitioners in various fields.

Statistical Mechanics,Chaotic Dynamics,Data Analysis, Statistics and Probability
Maximum entropy estimation of transition probabilities of reversible Markov chains

Erik Van der Straeten

DOI: https://doi.org/10.3390/e11040867

2009-11-25

Abstract:In this paper, we develop a general theory for the estimation of the transition probabilities of reversible Markov chains using the maximum entropy principle. A broad range of physical models can be studied within this approach. We use one-dimensional classical spin systems to illustrate the theoretical ideas. The examples studied in this paper are: the Ising model, the Potts model and the Blume-Emery-Griffiths model.

Statistical Mechanics
Entropy of absorbed Markov chains

Servet Martínez

DOI: https://doi.org/10.48550/arXiv.1910.00990

2019-10-02

Abstract:We consider a strictly substochastic matrix or an stochastic matrix with absorbing states. By using quasi-stationary distributions one shows there is a canonical associated stationary Markov chain. Based upon $2-$stringing representation of the resurrected chain we supply a stationary representation of the killed and the absorbed chains. The entropies of these representations has a clear meaning when one identifies the probability measure of natural factors. The balance between the entropies of these representations and the canonical chain, serves to check the suitability of the whole construction.

Probability
An estimator of entropy production for partially accessible Markov networks based on the observation of blurred transitions

Benjamin Ertel,Udo Seifert

DOI: https://doi.org/10.1103/PhysRevE.109.054109

2024-03-05

Abstract:A central task in stochastic thermodynamics is the estimation of entropy production for partially accessible Markov networks. We establish an effective transition-based description for such networks with transitions that are not distinguishable and therefore blurred for an external observer. We demonstrate that, in contrast to a description based on fully resolved transitions, this effective description is typically non-Markovian at any point in time. Starting from an information-theoretic bound, we derive an operationally accessible entropy estimator for this observation scenario. We illustrate the operational relevance and the quality of this entropy estimator with a numerical analysis of various representative examples.

Statistical Mechanics
Asymptotic Approximation to the Entropy Rate of Binary Hidden Markov Processes

CHEN Shuang-ping,ZHENG Hao-ran,TONG Qing,WANG Xu-fa

DOI: https://doi.org/10.3321/j.issn:0372-2112.2006.03.026

2006-01-01

Abstract:Based on the convergence of bounds for the entropy rate of binary hidden Markov processes,a numerical approach is advanced.The algorithm can approximate to true value of the entropy rate below a predefined error,and the accuracy can also be estimated.Since the logarithm of the algorithm′s complexity is linear to the logarithm of error,the cost of the algorithm is acceptable for practical use in engineering fields.It casts light to solve the problem of computing the entropy rate of more generalized kinds of HMPs.
Reversibility and Entropy Production of Inhomogeneous Markov Chains

Hao Ge,Da-Quan Jiang,Min Qian

DOI: https://doi.org/10.1239/jap/1165505205

2006-01-01

Abstract:In this paper we introduce the concepts of instantaneous reversibility and instantaneous entropy production rate for inhomogeneous Markov chains with denumerable state spaces. The following statements are proved to be equivalent: the inhomogeneous Markov chain is instantaneously reversible; it is in detailed balance; its entropy production rate vanishes. In particular, for a time-periodic birth-death chain, which can be regarded as a simple version of a physical model (Brownian motors), we prove that its rotation number is 0 when it is instantaneously reversible or periodically reversible. Hence, in our model of Markov chains, the directed transport phenomenon of Brownian motors can occur only in nonequilibrium and irreversible systems.
Estimator of entropy production for partially accessible Markov networks based on the observation of blurred transitions

Benjamin Ertel,Udo Seifert

DOI: https://doi.org/10.1103/physreve.109.054109

IF: 2.707

2024-05-07

Physical Review E

Abstract:A central task in stochastic thermodynamics is the estimation of entropy production for partially accessible Markov networks. We establish an effective transition-based description for such networks with transitions that are not distinguishable and therefore blurred for an external observer. We demonstrate that, in contrast to a description based on fully resolved transitions, this effective description is typically non-Markovian at any point in time. Starting from an information-theoretic bound, we derive an operationally accessible entropy estimator for this observation scenario. We illustrate the operational relevance and the quality of this entropy estimator with a numerical analysis of various representative examples. https://doi.org/10.1103/PhysRevE.109.054109 ©2024 American Physical Society

physics, fluids & plasmas, mathematical
Cutoff for mixtures of permuted Markov chains: general case

Bastien Dubail

2024-02-06

Abstract:We investigate the mixing properties of a finite Markov chain in random environment defined as a mixture of a deterministic chain and a chain whose state space has been permuted uniformly at random. This work is the counterpart of a companion paper where we focused on a reversible model, which allowed for a few simplifications in the proof. We consider here the general case. Under mild assumptions on the base Markov chains, we prove that with high probability the resulting chain exhibits the cutoff phenomenon at entropic time $\log n / h$, $h$ being some constant related to the entropy of the chain, when the chain is started from a typical state. However contrary to the reversible case uniform cutoff at entropic time does not hold, as we provide an example where the worst-case mixing time has at least polylogarithmic order. We also provide a polylogarithmic upper bound on the worst-case mixing time, which in fact plays a crucial role in deriving the main result for typical states. Incidentally, our proof gives a clear picture of when to expect uniform cutoff at entropic time: it appears as the consequence of a uniform transience property for the covering Markov chain used throughout the proof, which lies on an infinite state space and projects back onto the initial chain.

Probability
On the Entropy and Filtering of Hidden Markov Processes Observed Via Arbitrary Channels

Jun Luo,Dongning Guo

2008-01-01

Abstract:This paper studies the entropy and filtering of hidden Markov processes (HMPs) which are observations of discrete-time binary homogeneous Markov chains through an arbitrary memoryless channel. A fixed-point functional equation is derived for the stationary distribution of an input symbol conditioned on all past observations. While the existence of a solution to this equation is guaranteed by martingale theory, its uniqueness follows from contraction mapping property. In order to compute this distribution, the fixed-point functional equation is firstly converted to a linear system through quantization and then solved numerically using quadratic programming. The entropy or differential entropy rate of the HMP can be computed in two ways: one by exploiting the average entropy of each input symbol conditioned on past observations, and the other by applying a relationship between the input-output mutual information and the stationary distribution obtained via filtering. Two examples, where the numerical method is applied to the binary symmetric channel (BSC) and additive white Gaussian noise (AWGN) channel, are presented. Unlike many other numerical methods, this numerical solution is not based on averaging over a long sample path of the HMP.
Permutation Complexity and Coupling Measures in Hidden Markov Models

Taichi Haruna,Kohei Nakajima

DOI: https://doi.org/10.48550/arXiv.1204.1821

2012-04-09

Chaotic Dynamics

Abstract:In [Haruna, T. and Nakajima, K., 2011. Physica D 240, 1370-1377], the authors introduced the duality between values (words) and orderings (permutations) as a basis to discuss the relationship between information theoretic measures for finite-alphabet stationary stochastic processes and their permutation analogues. It has been used to give a simple proof of the equality between the entropy rate and the permutation entropy rate for any finite-alphabet stationary stochastic process and show some results on the excess entropy and the transfer entropy for finite-alphabet stationary ergodic Markov processes. In this paper, we extend our previous results to hidden Markov models and show the equalities between various information theoretic complexity and coupling measures and their permutation analogues. In particular, we show the following two results within the realm of hidden Markov models with ergodic internal processes: the two permutation analogues of the transfer entropy, the symbolic transfer entropy and the transfer entropy on rank vectors, are both equivalent to the transfer entropy if they are considered as the rates, and the directed information theory can be captured by the permutation entropy approach.
Inferring kinetics and entropy production from observable transitions in partially accessible, periodically driven Markov networks

Alexander M. Maier,Julius Degünther,Jann van der Meer,Udo Seifert

2024-06-18

Abstract:For a network of discrete states with a periodically driven Markovian dynamics, we develop an inference scheme for an external observer who has access to some transitions. Based on waiting-time distributions between these transitions, the periodic probabilities of states connected by these observed transitions and their time-dependent transition rates can be inferred. Moreover, the smallest number of hidden transitions between accessible ones and some of their transition rates can be extracted. We prove and conjecture lower bounds on the total entropy production for such periodic stationary states. Even though our techniques are based on generalizations of known methods for steady states, we obtain original results for those as well.

Statistical Mechanics
Entropy estimation for partially accessible Markov networks based on imperfect observations: Role of finite resolution and finite statistics

Jonas H. Fritz,Benjamin Ertel,Udo Seifert

2024-12-05

Abstract:Estimating entropy production from real observation data can be difficult due to finite resolution in both space and time and finite measurement statistics. We characterize the statistical error introduced by finite sample size and compare the performance of three different entropy estimators under these limitations for two different paradigmatic systems, a four-state Markov network and an augmented Michaelis-Menten reaction scheme. We consider the thermodynamic uncertainty relation, a waiting-time based estimator for resolved transitions and a waiting-time based estimator for blurred transitions in imperfect observation scenarios. For perfect measurement statistics and finite temporal resolution, the estimator based on resolved transitions performs best in all considered scenarios. The thermodynamic uncertainty relation gives a better estimate than the estimator based on blurred transitions at low driving affinities, whereas the latter performs better at high driving affinities. Furthermore, we find that a higher temporal and spatial resolution leads to slower convergence of measurement statistics, implying that for short measurement times, a lower resolution may be beneficial. Additionally, we identify a self-averaging effect for the waiting-time based entropy estimators that can reduce their variance for observations with finite statistics.

Statistical Mechanics
Cutoff for mixtures of permuted Markov chains: reversible case

Bastien Dubail

2024-02-08

Abstract:We investigate the mixing properties of a model of reversible Markov chains in random environment, which notably contains the simple random walk on the superposition of a deterministic graph and a second graph whose vertex set has been permuted uniformly at random. It generalizes in particular a result of Hermon, Sly and Sousi, who proved the cutoff phenomenon at entropic time for the simple random walk on a graph with an added uniform matching. Under mild assumptions on the base Markov chains, we prove that with high probability the resulting chain exhibits the cutoff phenomenon at entropic time log n/h, h being some constant related to the entropy of the chain. We note that the results presented here are the consequence of a work conducted for a more general model that does not assume reversibility, which will be the object of a companion paper. Thus, most of our proofs do not actually require reversibility, which constitutes an important technical contribution. Finally, our argument relies on a novel concentration result for "low-degree" functions on the symmetric group, established specifically for our purpose but which could be of independent interest.

Probability
The noise intensity of a Markov chain

Lukas Ramlow,Benjamin Lindner

2024-02-16

Abstract:Stochastic transitions between discrete microscopic states play an important role in many physical and biological systems. Often, these transitions lead to fluctuations on a macroscopic scale. A classic example from neuroscience is the stochastic opening and closing of ion channels and the resulting fluctuations in membrane current. When the microscopic transitions are fast, the macroscopic fluctuations are nearly uncorrelated and can be fully characterized by their mean and noise intensity. We show how, for an arbitrary Markov chain, the noise intensity can be determined from an algebraic equation, based on the transition rate matrix. We demonstrate the validity of the theory using an analytically tractable two-state Markovian dichotomous noise, an eight-state model for a Calcium channel subunit (De Young-Keizer model), and Markov models of the voltage-gated Sodium and Potassium channels as they appear in a stochastic version of the Hodgkin-Huxley model.

Statistical Mechanics,Data Analysis, Statistics and Probability

Entropy Rate for Hidden Markov Chains with rare transitions

Shannon Entropy Rate of Hidden Markov Processes

Some Generalized Entropy Ergodic Theorems for Nonhomogeneous Hidden Markov Models

Analyticity of Entropy Rate of Hidden Markov Chains

An efficient algorithm for the entropy rate of a hidden Markov model with unambiguous symbols

On the Entropy Rate of Hidden Markov Processes Observed Through Arbitrary Memoryless Channels

Entropy estimators for Markovian sequences: A comparative analysis

Maximum entropy estimation of transition probabilities of reversible Markov chains

Entropy of absorbed Markov chains

An estimator of entropy production for partially accessible Markov networks based on the observation of blurred transitions

Asymptotic Approximation to the Entropy Rate of Binary Hidden Markov Processes

Reversibility and Entropy Production of Inhomogeneous Markov Chains

Estimator of entropy production for partially accessible Markov networks based on the observation of blurred transitions

Cutoff for mixtures of permuted Markov chains: general case

On the Entropy and Filtering of Hidden Markov Processes Observed Via Arbitrary Channels

Permutation Complexity and Coupling Measures in Hidden Markov Models

Inferring kinetics and entropy production from observable transitions in partially accessible, periodically driven Markov networks

Entropy estimation for partially accessible Markov networks based on imperfect observations: Role of finite resolution and finite statistics

Cutoff for mixtures of permuted Markov chains: reversible case

The noise intensity of a Markov chain