Ziv-Merhav estimation for hidden-Markov processes

Nicholas Barnfield,Raphaël Grondin,Gaia Pozzoli,Renaud Raquépas

2024-08-16

Abstract:We present a proof of strong consistency of a Ziv-Merhav-type estimator of the cross entropy rate for pairs of hidden-Markov processes. Our proof strategy has two novel aspects: the focus on decoupling properties of the laws and the use of tools from the thermodynamic formalism.

Information Theory,Probability

What problem does this paper attempt to address?

On the Ziv-Merhav theorem beyond Markovianity II: leveraging the thermodynamic formalism

Nicholas Barnfield,Raphaël Grondin,Gaia Pozzoli,Renaud Raquépas

2024-03-25

Abstract:We prove asymptotic results for a modification of the cross-entropy estimator originally introduced by Ziv and Merhav in the Markovian setting in 1993. Our results concern a more general class of decoupled measures on shift spaces over a finite alphabet and in particular imply strong asymptotic consistency of the modified estimator for all pairs of functions of stationary, irreducible, finite-state Markov chains satisfying a mild decay condition. Our approach is based on the study of a rescaled cumulant-generating function called the cross-entropic pressure, importing to information theory some techniques from the study of large deviations within the thermodynamic formalism.

Probability,Information Theory,Dynamical Systems
On the Ziv–Merhav theorem beyond Markovianity I

Nicholas Barnfield,Raphaël Grondin,Gaia Pozzoli,Renaud Raquépas

DOI: https://doi.org/10.4153/s0008414x24000178

2024-03-07

Canadian Journal of Mathematics

Abstract:Abstract We generalize to a broader class of decoupled measures a result of Ziv and Merhav on universal estimation of the specific cross (or relative) entropy, originally for a pair of multilevel Markov measures. Our generalization focuses on abstract decoupling conditions and covers pairs of suitably regular g-measures and pairs of equilibrium measures arising from the “small space of interactions” in mathematical statistical mechanics.

mathematics
A Donsker-type Theorem for Log-likelihood Processes

Zhonggen Su,Hanchao Wang

DOI: https://doi.org/10.1007/s10959-019-00926-9

2019-01-01

Journal of Theoretical Probability

Abstract:Let $$(\Omega , \mathcal {F}, (\mathcal {F})_{t\ge 0}, P)$$ be a complete stochastic basis, and X be a semimartingale with predictable compensator $$(B, C, \nu )$$. Consider a family of probability measures $$\mathbf {P}=( {P}^{n, \psi }, \psi \in \Psi , n\ge 1)$$, where $$\Psi $$ is an index set, $$ {P}^{n, \psi }{\mathop {\ll }\limits ^\mathrm{loc}}{P}$$, and denote the likelihood ratio process by $$Z_t^{n, \psi } =\frac{\mathrm{d}P^{n, \psi }|_{\mathcal {F}_t}}{\mathrm{d} P|_{\mathcal {F}_t}}$$. Under some regularity conditions in terms of logarithm entropy and Hellinger processes, we prove that $$\log Z_t^{n}$$ converges weakly to a Gaussian process in $$\ell ^\infty (\Psi )$$ as $$n\rightarrow \infty $$ for each fixed $$t>0$$.
On Maximum a Posteriori Estimation of Hidden Markov Processes

Armen Allahverdyan,Aram Galstyan

DOI: https://doi.org/10.48550/arXiv.0906.1980

2009-06-11

Abstract:We present a theoretical analysis of Maximum a Posteriori (MAP) sequence estimation for binary symmetric hidden Markov processes. We reduce the MAP estimation to the energy minimization of an appropriately defined Ising spin model, and focus on the performance of MAP as characterized by its accuracy and the number of solutions corresponding to a typical observed sequence. It is shown that for a finite range of sufficiently low noise levels, the solution is uniquely related to the observed sequence, while the accuracy degrades linearly with increasing the noise strength. For intermediate noise values, the accuracy is nearly noise-independent, but now there are exponentially many solutions to the estimation problem, which is reflected in non-zero ground-state entropy for the Ising model. Finally, for even larger noise intensities, the number of solutions reduces again, but the accuracy is poor. It is shown that these regimes are different thermodynamic phases of the Ising model that are related to each other via first-order phase transitions.

Artificial Intelligence,Statistical Mechanics,Information Theory,Data Analysis, Statistics and Probability,Machine Learning
Consistency of likelihood estimation for Gibbs point processes

David Dereudre,Frédéric Lavancier

DOI: https://doi.org/10.48550/arXiv.1506.02887

2016-01-26

Abstract:Strong consistency of the maximum likelihood estimator (MLE) for parametric Gibbs point process models is established. The setting is very general. It includes pairwise pair potentials, finite and infinite multibody interactions and geometrical interactions, where the range can be finite or infinite. The Gibbs interaction may depend linearly or non-linearly on the parameters, a particular case being hardcore parameters and interaction range parameters. As important examples, we deduce the consistency of the MLE for all parameters of the Strauss model, the hardcore Strauss model, the Lennard-Jones model and the area-interaction model.

Statistics Theory
Hidden temperature in the KMP model

Anna De Masi,Pablo A. Ferrari,Davide Gabrielli

2024-06-03

Abstract:In the Kipnis Marchioro Presutti (KMP) model a positive energy $\zeta_i$ is associated with each vertex $i$ of a finite graph with a boundary. When a Poisson clock rings at an edge $ij$ with energies $\zeta_i,\zeta_j$, those values are substituted by $U(\zeta_i+\zeta_j)$ and $(1-U)(\zeta_i+\zeta_j)$, respectively, where $U$ is a uniform random variable in $(0,1)$. A value $T_j\ge 0$ is fixed at each boundary vertex $j$. The dynamics is defined in such way that the resulting Markov process $\zeta(t)$, satisfies that $\zeta_j(t)$ is exponential with mean $T_j$, for each boundary vertex $j$, for all $t$. We show that the invariant measure is the distribution of a vector $\zeta$ with coordinates $\zeta_i=T_i X_i$, where $X_i$ are iid exponential$(1)$ random variables, the law of $T$ is the invariant measure for an opinion random averaging/gossip model with the same boundary conditions of $\zeta$, and the vectors $X$ and $T$ are independent. The result confirms a conjecture based on the large deviations of the model. When the graph is one-dimensional, we bound the correlations of the invariant measure and perform the hydrostatic limit. We show that the empirical measure of a configuration chosen with the invariant measure converges to the linear interpolation of the boundary values.

Probability,Statistical Mechanics
Maximum Entropy Estimator for Hidden Markov Models: Reduction to Dimension 2

Shulan Hu,Xinyu Wang,Liming Wu

2023-02-19

Abstract:In the paper, we introduce the maximum entropy estimator based on 2-dimensional empirical distribution of the observation sequence of hidden Markov model , when the sample size is big: in that case computing the maximum likelihood estimator is too consuming in time by the classical Baum-Welch EM algorithm. We prove the consistency and the asymptotic normality of the maximum entropy estimator in a quite general framework, where the asymptotic covariance matrix is explicitly estimated in terms of the 2-dimensional Fisher information. To complement it, the 2-dimensional relative entropy is skillfully used to study the hypotheses testing problem. Furthermore, we propose 2-dimensional maximum entropy algorithm for finding the maximum entropy estimator, which works for very large observation dataset and large hidden states set. Some numerical examples are furnished and commented for illustrating our theoretical results.

Statistics Theory,Probability
Large deviations of return times and related entropy estimators on shift spaces

Noé Cuneo,Renaud Raquépas

2024-05-22

Abstract:We prove the large deviation principle for several entropy and cross entropy estimators based on return times and waiting times on shift spaces over finite alphabets. We consider shift-invariant probability measures satisfying some decoupling conditions which imply no form of mixing nor ergodicity. We establish precise relations between the rate functions of the different estimators, and between these rate functions and the corresponding pressures, one of which is the Rényi entropy function. For the most commonly used definition of return times, the large-deviation rate function is proved to be nonconvex, except in marginal cases. The results apply in particular to irreducible Markov chains, equilibrium measures for Bowen-regular potentials, g-measures, invariant Gibbs states for absolutely summable interactions in statistical mechanics, and also to probability measures which may be far from Gibbsian, including some hidden Markov models and repeated quantum measurement processes.

Probability,Mathematical Physics,Dynamical Systems
Thermodynamic inference in partially accessible Markov networks: A unifying perspective from transition-based waiting time distributions

Jann van der Meer,Benjamin Ertel,Udo Seifert

DOI: https://doi.org/10.1103/PhysRevX.12.031025

2022-08-18

Abstract:The inference of thermodynamic quantities from the description of an only partially accessible physical system is a central challenge in stochastic thermodynamics. A common approach is coarse-graining, which maps the dynamics of such a system to a reduced effective one. While coarse-graining states of the system into compound ones is a well studied concept, recent evidence hints at a complementary description by considering observable transitions and waiting times. In this work, we consider waiting time distributions between two consecutive transitions of a partially observable Markov network. We formulate an entropy estimator using their ratios to quantify irreversibility. Depending on the complexity of the underlying network, we formulate criteria to infer whether the entropy estimator recovers the full physical entropy production or whether it just provides a lower bound that improves on established results. This conceptual approach, which is based on the irreversibility of underlying cycles, additionally enables us to derive estimators for the topology of the network, i.e., the presence of a hidden cycle, its number of states and its driving affinity. Adopting an equivalent semi-Markov description, our results can be condensed into a fluctuation theorem for the corresponding semi-Markov process. This mathematical perspective provides a unifying framework for the entropy estimators considered here and established earlier ones. The crucial role of the correct version of time-reversal helps to clarify a recent debate on the meaning of formal versus physical irreversibility. Extensive numerical calculations based on a direct evaluation of waiting-time distributions illustrate our exact results and provide an estimate on the quality of the bounds for affinities of hidden cycles.

Statistical Mechanics
Consistency of the maximum likelihood estimator in hidden Markov models with trends

Luc Lehéricy,Augustin Touron

DOI: https://doi.org/10.48550/arXiv.2112.08731

2021-12-16

Abstract:A hidden Markov model with trends is a hidden Markov model whose emission distributions are translated by a trend that depends on the current hidden state and on the current time. Contrary to standard hidden Markov models, such processes are not homogeneous and cannot be made homogeneous by a simple de-trending step. We show that when the trends are polynomial, the maximum likelihood estimator is able to recover the trends together with the other parameters and is strongly consistent. More precisely, the supremum norm of the difference between the true trends and the estimated ones tends to zero. Numerical properties of the maximum likelihood estimator are assessed by a simulation study.

Statistics Theory
The Bernstein-von Mises theorem for Semiparametric Mixtures

Stefan Franssen,Jeanne Nguyen,Aad van der Vaart

2024-11-30

Abstract:Semiparametric mixture models are parametric models with latent variables. They are defined kernel, $p_\theta(x | z)$, where z is the unknown latent variable, and $\theta$ is the parameter of interest. We assume that the latent variables are an i.i.d. sample from some mixing distribution $F$. A Bayesian would put a prior on the pair $(\theta, F)$. We prove consistency for these models in fair generality and then study efficiency. We first prove an abstract Semiparametric Bernstein-von Mises theorem, and then provide tools to verify the assumptions. We use these tools to study the efficiency for estimating $\theta$ in the frailty model and the errors in variables model in the case were we put a generic prior on $\theta$ and a species sampling process prior on $F$.

Statistics Theory
Maximum likelihood estimation in hidden Markov models with inhomogeneous noise

Manuel Diehn,Axel Munk,Daniel Rudolf

2018-04-11

Abstract:We consider parameter estimation in finite hidden state space Markov models with time-dependent inhomogeneous noise, where the inhomogeneity vanishes sufficiently fast. Based on the concept of asymptotic mean stationary processes we prove that the maximum likelihood and a quasi-maximum likelihood estimator (QMLE) are strongly consistent. The computation of the QMLE ignores the inhomogeneity, hence, is much simpler and robust. The theory is motivated by an example from biophysics and applied to a Poisson- and linear Gaussian model.

Methodology,Statistics Theory
Asymptotic properties of the maximum likelihood estimator for Hidden Markov Models indexed by binary trees

Julien Weibel

2024-09-10

Abstract:We consider hidden Markov models indexed by a binary tree where the hidden state space is a general metric space. We study the maximum likelihood estimator (MLE) of the model parameters based only on the observed variables. In both stationary and non-stationary regimes, we prove strong consistency and asymptotic normality of the MLE under standard assumptions. Those standard assumptions imply uniform exponential memorylessness properties of the initial distribution conditional on the observations. The proofs rely on ergodic theorems for Markov chain indexed by trees with neighborhood-dependent functions.

Probability,Statistics Theory
Hörmander properties of discrete time Markov processes

Clément Rey

2024-03-19

Abstract:We present an abstract framework for establishing smoothing properties within a specific class of inhomogeneous discrete-time Markov processes. These properties, in turn, serve as a basis for demonstrating the existence of density functions for our processes or more precisely for regularized versions of them. They can also be exploited to show the total variation convergence towards the solution of a Stochastic Differential Equation as the time step between two observations of the discrete time Markov processes tends to zero. The distinctive feature of our methodology lies in the exploration of smoothing properties under some local weak Hörmander type conditions satisfied by the discrete-time Markov processes. Our Hörmander properties are demonstrated to align with the standard local weak Hörmander properties satisfied by the coefficients of the Stochastic Differential Equations which are the total variation limits of our discrete time Markov processes.

Probability
Intertwining and propagation of mixtures for generalized KMP models and harmonic models

Cristian Giardinà,Frank Redig,Berend van Tol

2024-06-03

Abstract:We study a class of stochastic models of mass transport on discrete vertex set $V$. For these models, a one-parameter family of homogeneous product measures $\otimes_{i\in V} \nu_\theta$ is reversible. We prove that the set of mixtures of inhomogeneous product measures with equilibrium marginals, i.e., the set of measures of the form \[ \int\Big(\bigotimes_{i\in V} \nu_{\theta_i}\Big) \,\Xi(\prod_{i\in V}d\theta_i) \] is left invariant by the dynamics in the course of time, and the ``mixing measure'' $\Xi$ evolves according to a Markov process which we then call ``the hidden parameter model''. This generalizes results from [7] to a larger class of models and on more general graphs. The class of models includes discrete and continuous generalized KMP models, as well as discrete and continuous harmonic models. The results imply that in all these models, the non-equilibrium steady state of their reservoir driven version is a mixture of product measures where the mixing measure is in turn the stationary state of the corresponding ``hidden parameter model''. For the boundary-driven harmonic models on the chain $\{1,\ldots, N\}$ with nearest neighbor edges, we recover that the stationary measure of the hidden parameter model is the joint distribution of the ordered Dirichlet distribution (cf. [3]), with a purely probabilistic proof based on a spatial Markov property of the hidden parameter model.

Probability,Statistical Mechanics,Mathematical Physics
Rate-optimal estimation of mixed semimartingales

Carsten H. Chong,Thomas Delerue,Fabian Mies

2024-10-25

Abstract:Consider the sum $Y=B+B(H)$ of a Brownian motion $B$ and an independent fractional Brownian motion $B(H)$ with Hurst parameter $H\in(0,1)$. Even though $B(H)$ is not a semimartingale, it was shown in [\textit{Bernoulli} \textbf{7} (2001) 913--934] that $Y$ is a semimartingale if $H>3/4$. Moreover, $Y$ is locally equivalent to $B$ in this case, so $H$ cannot be consistently estimated from local observations of $Y$. This paper pivots on another unexpected feature in this model: if $B$ and $B(H)$ become correlated, then $Y$ will never be a semimartingale, and $H$ can be identified, regardless of its value. This and other results will follow from a detailed statistical analysis of a more general class of processes called \emph{mixed semimartingales}, which are semiparametric extensions of $Y$ with stochastic volatility in both the martingale and the fractional component. In particular, we derive consistent estimators and feasible central limit theorems for all parameters and processes that can be identified from high-frequency observations. We further show that our estimators achieve optimal rates in a minimax sense.

Statistics Theory,Probability,Methodology
Provable Statistical Rates for Consistency Diffusion Models

Zehao Dou,Minshuo Chen,Mengdi Wang,Zhuoran Yang

2024-06-24

Abstract:Diffusion models have revolutionized various application domains, including computer vision and audio generation. Despite the state-of-the-art performance, diffusion models are known for their slow sample generation due to the extensive number of steps involved. In response, consistency models have been developed to merge multiple steps in the sampling process, thereby significantly boosting the speed of sample generation without compromising quality. This paper contributes towards the first statistical theory for consistency models, formulating their training as a distribution discrepancy minimization problem. Our analysis yields statistical estimation rates based on the Wasserstein distance for consistency models, matching those of vanilla diffusion models. Additionally, our results encompass the training of consistency models through both distillation and isolation methods, demystifying their underlying advantage.

Machine Learning
Estimating the Mixing Coefficients of Geometrically Ergodic Markov Processes

Steffen Grünewälder,Azadeh Khaleghi

2024-02-12

Abstract:We propose methods to estimate the individual $\beta$-mixing coefficients of a real-valued geometrically ergodic Markov process from a single sample-path $X_0,X_1, \dots,X_n$. Under standard smoothness conditions on the densities, namely, that the joint density of the pair $(X_0,X_m)$ for each $m$ lies in a Besov space $B^s_{1,\infty}(\mathbb R^2)$ for some known $s>0$, we obtain a rate of convergence of order $\mathcal{O}(\log(n) n^{-[s]/(2[s]+2)})$ for the expected error of our estimator in this case\footnote{We use $[s]$ to denote the integer part of the decomposition $s=[s]+\{s\}$ of $s \in (0,\infty)$ into an integer term and a {\em strictly positive} remainder term $\{s\} \in (0,1]$.}. We complement this result with a high-probability bound on the estimation error, and further obtain analogues of these bounds in the case where the state-space is finite. Naturally no density assumptions are required in this setting; the expected error rate is shown to be of order $\mathcal O(\log(n) n^{-1/2})$.

Statistics Theory,Machine Learning
Persistence problems for additive functionals of one-dimensional Markov processes

Quentin Berger,Loïc Béthencourt,Camille Tardif

2023-04-18

Abstract:In this article, we consider additive functionals $\zeta_t = \int_0^t f(X_s)\mathrm{d} s$ of a càdlàg Markov process $(X_t)_{t\geq 0}$ on $\mathbb{R}$. Under some general conditions on the process $(X_t)_{t\geq 0}$ and on the function $f$, we show that the persistence probabilities verify $\mathbb{P}(\zeta_s < z \text{ for all } s\leq t ) \sim \mathcal{V}(z) \varsigma(t) t^{-\theta}$ as $t\to\infty$, for some (explicit) $\mathcal{V}(\cdot)$, some slowly varying function $\varsigma(\cdot)$ and some $\theta\in (0,1)$. This extends results in the literature, which mostly focused on the case of a self-similar process $(X_t)_{t\geq 0}$ (such as Brownian motion or skew-Bessel process) with a homogeneous functional $f$ (namely a pure power, possibly asymmetric). In a nutshell, we are able to deal with processes which are only asymptotically self-similar and functionals which are only asymptotically homogeneous. Our results rely on an excursion decomposition of $(X_t)_{t\geq 0}$, together with a Wiener--Hopf decomposition of an auxiliary (bivariate) Lévy process, with a probabilistic point of view. This provides an interpretation for the asymptotic behavior of the persistence probabilities, and in particular for the exponent $\theta$, which we write as $\theta = \rho \beta$, with $\beta$ the scaling exponent of the local time of $(X_{t})_{t\geq 0}$ at level $0$ and $\rho$ the (asymptotic) positivity parameter of the auxiliary Lévy process.

Probability
Some Generalized Entropy Ergodic Theorems for Nonhomogeneous Hidden Markov Models

Qifeng Yao,Longsheng Cheng,Wenhe Chen,Ting Mao

DOI: https://doi.org/10.3390/math12040605

IF: 2.4

2024-02-18

Mathematics

Abstract:Entropy measures the randomness or uncertainty of a stochastic process, and the entropy rate refers to the limit of the time average of entropy. The generalized entropy rate in the form of delayed averages can overcome the redundancy of initial information while ensuring stationarity. Therefore, it has better practical value. A Hidden Markov Model (HMM) contains two stochastic processes, a stochastic process in which all states can be observed and a Markov chain in which all states cannot be observed. The entropy rate is an important characteristic of HMMs. The transition matrix of a homogeneous HMM is unique, while a Nonhomogeneous Hidden Markov Model (NHMM) requires the transition matrices to be dependent on time variables. From the perspective of model structure, NHMMs are novel extensions of homogeneous HMMs. In this paper, the concepts of the generalized entropy rate and NHMMs are defined and fully explained, a strong limit theorem and limit properties of a norm are presented, and then generalized entropy ergodic theorems with an almost surely convergence for NHMMs are obtained. These results provide concise formulas for the computation and estimation of the generalized entropy rate for NHMMs.

mathematics

Ziv-Merhav estimation for hidden-Markov processes

On the Ziv-Merhav theorem beyond Markovianity II: leveraging the thermodynamic formalism

On the Ziv–Merhav theorem beyond Markovianity I

A Donsker-type Theorem for Log-likelihood Processes

On Maximum a Posteriori Estimation of Hidden Markov Processes

Consistency of likelihood estimation for Gibbs point processes

Hidden temperature in the KMP model

Maximum Entropy Estimator for Hidden Markov Models: Reduction to Dimension 2

Large deviations of return times and related entropy estimators on shift spaces

Thermodynamic inference in partially accessible Markov networks: A unifying perspective from transition-based waiting time distributions

Consistency of the maximum likelihood estimator in hidden Markov models with trends

The Bernstein-von Mises theorem for Semiparametric Mixtures

Maximum likelihood estimation in hidden Markov models with inhomogeneous noise

Asymptotic properties of the maximum likelihood estimator for Hidden Markov Models indexed by binary trees

Hörmander properties of discrete time Markov processes

Intertwining and propagation of mixtures for generalized KMP models and harmonic models

Rate-optimal estimation of mixed semimartingales

Provable Statistical Rates for Consistency Diffusion Models

Estimating the Mixing Coefficients of Geometrically Ergodic Markov Processes

Persistence problems for additive functionals of one-dimensional Markov processes

Some Generalized Entropy Ergodic Theorems for Nonhomogeneous Hidden Markov Models