2 Circulation Distribution, Entropy Production and Irreversibility of Finite Markov Chains with Continuous Parameter

Da-Quan Jiang,Min Qian,Min-Ping Qian

DOI: https://doi.org/10.1007/978-3-540-40957-1_3

2004-01-01

Abstract: • 2.1 Circulation Distribution • 2.2 Irreversibility and Entropy Production • 2.3 Large Deviations and Fluctuation Theorem

What problem does this paper attempt to address?

Circulation Distribution, Entropy Production And Irreversibility Of Denumerable Markov Chains

Da-Quan Jiang,Min Qian,Min-Ping Qian

DOI: https://doi.org/10.1007/978-3-540-40957-1_2

2004-01-01

Abstract: • 1.1 Directed Circuits, Cycles and Passage Functions • 1.2 The Derived Chain • 1.3 Circulation Distribution of Recurrent Markov Chains • 1.4 Irreversibility and Entropy Production • 1.5 Large Deviations and Fluctuation Theorem • 1.6 Appendix
Thermodynamics of Markov Processes with Non-extensive Entropy and Free Energy

Liangrong Peng,Hong Qian,Liu Hong

DOI: https://doi.org/10.1103/PhysRevE.101.022114

2020-03-31

Abstract:Statistical thermodynamics of small systems shows dramatic differences from normal systems. Parallel to the recently presented steady-state thermodynamic formalism for master equation and Fokker-Planck equation, we show that a ``thermodynamic'' theory can also be developed based on Tsallis' generalized entropy $S^{(q)}=\sum_{i=1}^N(p_i-p_i^q)/[q(q-1)]$ and Shiino's generalized free energy $F^{(q)}=[\sum_{i=1}^Np_i(p_i/\pi_i)^{q-1}-1]/[q(q-1)]$, where $\pi_i$ is the stationary distribution. $dF^{(q)}/dt=-f_d^{(q)}\le 0$ and it is zero iff the system is in its stationary state. $dS^{(q)}/dt-Q_{ex}^{(q)} = f_d^{(q)}$ where $Q_{ex}^{(q)}$ characterizes the heat exchange. For systems approaching equilibrium with detailed balance, $f_d^{(q)}$ is the product of Onsager's thermodynamic flux and force. However, it is discovered that the Onsager's force is non-local. This is a consequence of the particular transformation invariance for zero energy of Tsallis' statistics.

Mathematical Physics
Circulations of Markov-Chains with Continuous-Time and the Probability Interpretation of Some Determinants

MP QIAN,M QIAN,C QIAN

1984-01-01

Abstract:In this paper, the relation between the irreversibility and circulation of a Markov chainwith continuous time parameter is considered. By the calculation of determinants, it showshow the probability approach in[1] is related to T. Hill's results.
Cycle Symmetries and Circulation Fluctuations for Discrete-Time and Continuous-Time Markov Chains

Chen Jia,Da-Quan Jiang,Min-Ping Qian

DOI: https://doi.org/10.1214/15-aap1152

2016-01-01

The Annals of Applied Probability

Abstract:In probability theory, equalities are much less than inequalities. In this paper, we find a series of equalities which characterize the symmetry of the forming times of a family of similar cycles for discrete-time and continuous-time Markov chains. Moreover, we use these cycle symmetries to study the circulation fluctuations for Markov chains. We prove that the empirical circulations of a family of cycles passing through a common state satisfy a large deviation principle with a rate function which has an highly non-obvious symmetry. Finally, we discuss the applications of our work in statistical physics and biochemistry.
5 Entropy Production, Rotation Numbers and Irreversibility of Diffusion Processes on Manifolds

Da-Quan Jiang,Min Qian,Min-Ping Qian

DOI: https://doi.org/10.1007/978-3-540-40957-1_6

2004-01-01

Abstract: • 5.1 A Simple Example: Diffusion Processes on a Circle • 5.2 Lifted Processes and Cameron-Martin-Girsanov Formula • 5.3 Irreversibility and Entropy Production • 5.4 Rotation Numbers and Circulation Decomposition
Reversibility and Entropy Production of Inhomogeneous Markov Chains

Hao Ge,Da-Quan Jiang,Min Qian

DOI: https://doi.org/10.1239/jap/1165505205

2006-01-01

Abstract:In this paper we introduce the concepts of instantaneous reversibility and instantaneous entropy production rate for inhomogeneous Markov chains with denumerable state spaces. The following statements are proved to be equivalent: the inhomogeneous Markov chain is instantaneously reversible; it is in detailed balance; its entropy production rate vanishes. In particular, for a time-periodic birth-death chain, which can be regarded as a simple version of a physical model (Brownian motors), we prove that its rotation number is 0 when it is instantaneously reversible or periodically reversible. Hence, in our model of Markov chains, the directed transport phenomenon of Brownian motors can occur only in nonequilibrium and irreversible systems.
A Simple Discrete Model of Brownian Motors: Time-periodic Markov Chains

Hao Ge,Da-Quan Jiang,Min Qian

DOI: https://doi.org/10.1007/s10955-006-9099-6

2006-01-01

Journal of Statistical Physics

Abstract:In this paper, we consider periodically inhomogeneous Markov chains, which can be regarded as a simple version of physical model—Brownian motors. We introduce for them the concepts of periodical reversibility, detailed balance, entropy production rate and circulation distribution. We prove the equivalence of the following statements: The time-periodic Markov chain is periodically reversible; It is in detailed balance; Kolmogorov's cycle condition is satisfied; Its entropy production rate vanishes; Every circuit and its reversed circuit have the same circulation weight. Hence, in our model of Markov chains, the directed transport phenomenon of Brownian motors, i.e. the existence of net circulation, can occur only in nonequilibrium and irreversible systems. Moreover, we verify the large deviation property and the Gallavotti-Cohen fluctuation theorem of sample entropy production rates of the Markov chain.
Large deviations and related problems for absorbing Markov chains

Jinwen Chen,Xiaoxue Deng

DOI: https://doi.org/10.1016/j.spa.2013.02.014

IF: 1.43

2013-01-01

Stochastic Processes and their Applications

Abstract:In this paper, large deviations and their connections with several other fundamental topics are investigated for absorbing Markov chains. A variational representation for the Dirichlet principal eigenvalues is given by the large deviation approach. Kingman’s decay parameters and mean ratio quasi-stationary distributions of the chains are also characterized by the large deviation rate function. As an application of these results, we interpret the “stationarity” of mean ratio quasi-stationary distributions via a concrete example. An application to quasi-ergodicity is also discussed.
The Green–Kubo formula, autocorrelation function and fluctuation spectrum for finite Markov chains with continuous time

Yong Chen,Xi Chen,Min-Ping Qian

DOI: https://doi.org/10.1088/0305-4470/39/11/001

2006-01-01

Abstract:A general form of the Green-Kubo formula, which describes the fluctuations pertaining to all the steady states whether equilibrium or non-equilibrium, for a system driven by a finite Markov chain with continuous time (briefly, MC)({xi(1)}, is shown. The equivalence of different forms of the Green-Kubo formula is exploited. We also look at the differences in terms of the autocorrelation function and the fluctuation spectrum between the equilibrium state and the non-equilibrium steady state. Also, if the MC is in the non-equilibrium steady state, we can always find a complex function p, such that the fluctuation spectrum of {p(xi(1))} is non-monotonous in [0,+infinity).
Advanced method and application on Markov state transfer chain

Xi Li,Xiaoning Zhu,Guoqiang Cai,Yuan Sun

2008-01-01

Abstract:This paper advanced the solution of Markov state transfer chain to analyze the system reliability. According to the conversion of mutual relations, the paper put forward a method to calculate the life expectancy of top events in Markov quantitative analysis. Afterwards the paper presented the failure mode, failure rate and life expectancy of the top events. Meanwhile, the paper presented the method to calculate probability of important bottom events to top events. Finally the research is verified in the analysis of a city rail traffic automatic train super-vision system.
The invariant meauures, probability flux and circulations of one-dimensional Markov processes

Gong Guang,Qian Minping

DOI: https://doi.org/10.1007/BFb0093042

1982-01-01

Abstract:Without Abstract
Irreversible Circulation of Fluctuation and Entropy Production

Hiroyuki Tomita,Mitsusada M. Sano

DOI: https://doi.org/10.1143/PTP.119.515

2008-04-16

Abstract:Physical and chemical stochastic processes described by the master equation are investigated. In this paper, we examine the entropy production both for the master equation and for the corresponding Fokker-Planck equation. For the master equation, the exact expression of the entropy production was recently derived by Gaspard using the Kolmogorov-Sinai entropy ({\em <a class="link-external link-http" href="http://J.Stat.Phys" rel="external noopener nofollow">this http URL</a>.}, \textbf{117} (2004), 599; [Errata; \textbf{126} (2006), 1109 ]). Although Gaspard's expression is derived from a stochastic consideration, it should be noted that Gaspard's expression conincides with the thermodynamical expression. For the corresponding Fokker-Planck equation, by using the detailed imbalance relation which appears in the derivation process of the fluctuation theorem through the Onsger-Machlup theory, the entropy production is expressed in terms of the {\em irreversible circulation of fluctuation}, which was proposed by Tomita and Tomita ({\em <a class="link-external link-http" href="http://Prog.Theor.Phys" rel="external noopener nofollow">this http URL</a>.}, \textbf{51} (1974), 1731). However, this expression for the corresponding Fokker-Planck equation differs from that of the entropy production for the master equation. This discrepancy is due to the difference between the master equation and the corresponding Fokker-Planck equation, namely the former treats discrete events, but the latter equation is an approximation of the former one. In fact, in the latter equation, the original discrete events are smoothed out. To overcome this difficulty, we propose the {\em path weight principle}. By using this principle, the modified expression of the entropy production for the corresponding Fokker-Planck equation coincides with that of the master equation (i.e., the thermodynamical expression) for a simple chemical reaction system and a diffusion system.

Statistical Mechanics
Canonical structure and orthogonality of forces and currents in irreversible Markov chains

Marcus Kaiser,Robert L. Jack,Johannes Zimmer

DOI: https://doi.org/10.1007/s10955-018-1986-0

2018-02-16

Abstract:We discuss a canonical structure that provides a unifying description of dynamical large deviations for irreversible finite state Markov chains (continuous time), Onsager theory, and Macroscopic Fluctuation Theory. For Markov chains, this theory involves a non-linear relation between probability currents and their conjugate forces. Within this framework, we show how the forces can be split into two components, which are orthogonal to each other, in a generalised sense. This splitting allows a decomposition of the pathwise rate function into three terms, which have physical interpretations in terms of dissipation and convergence to equilibrium. Similar decompositions hold for rate functions at level 2 and level 2.5. These results clarify how bounds on entropy production and fluctuation theorems emerge from the underlying dynamical rules. We discuss how these results for Markov chains are related to similar structures within Macroscopic Fluctuation Theory, which describes hydrodynamic limits of such microscopic models.c Fluctuation Theory, which describes hydrodynamic limits of such microscopic models.

Physics
Large Deviations for Markov Chains and Spectrum of Matrices

Jinwen Chen

DOI: https://doi.org/10.1360/n012019-00019

2019-01-01

Abstract:This paper deals with large deviations and related problems for Markov chains with countable state spaces. By comparing with some dominating chains that satisfy the large deviation principle, we weaken the requirement for irreducibility to get the large deviation principle. In this way we can also characterize the stationary distributions of the chains via the large deviation rate functions. In particular, we prove that any finite Markov chain satisfies the large deviation principle. The global and local minimums ofthe rate function are shown to be closely related to the principal eigenvalues and the corresponding left and right eigenfunctions of the transition matrix.
Representation and Characterization of Quasistationary Distributions for Markov Chains

Iddo Ben-Ari,Ningwei Jiang

2024-05-24

Abstract:This work provides complete description of Quasistationary Distributions (QSDs) for Markov chains with a unique absorbing state and an irreducible set of non-absorbing states. As is well-known, every QSD has an associated absorption parameter describing the exponential tail of the absorption time under the law of the process with the QSD as the initial distribution. The analysis associated with the existence and representation of QSDs corresponding to a given parameter is according to whether the moment generating function of the absorption time starting from any non-absorbing state evaluated at the parameter is finite or infinite, the finite or infinite moment generating function regimes, respectively. For parameters in the finite regime, it is shown that when exist, all QSDs are in the convex cone of a Martin entry boundary associated with the parameter. The infinite regime corresponds to at most one parameter value and at most one QSD. In this regime, when a QSD exists, it is unique and can be represented by a renewal-type formula. Multiple applications to the findings are presented, including revisiting some of the main classical results in the area.

Probability
Non-equilibrium Stochastic Dynamics

QIAN MinPing,JIANG DaQuan

DOI: https://doi.org/10.1360/n012017-00178

2017-01-01

Abstract:In this paper,we mainly review the basic concepts in statistical mechanics of stochastic processes,including entropy production,flux,potential,equilibrium state,temperature in stochastic processes and their mathematical definitions.Moreover,we review some basic properties of the processes derived from these concepts,including fluctuation theorems,Green-Kubo formula,and the monotonicity of power spectrum density.Especially,we point out that we can introduce the temperature parameter in a Markov process to get the multi-layer clustering of the system modeled by the process.This multi-layer clustering shows the interaction network structure among the states.Finally,we give some examples of application problems,in which one can model by stochastic processes to get the answer,explanation,and understanding of concerned problems in natural sciences.
On the Relation Between Reversibility and Monotonicity of Fluctuation Spectra for Discrete Time Finite State Markov Chains

Yong Chen,Min-Ping Qian,Han-Sheng Xie

DOI: https://doi.org/10.1016/j.spl.2008.01.095

IF: 0.718

2008-01-01

Statistics & Probability Letters

Abstract:For the irreducible aperiodic discrete time finite state Markov chain, (i) if it is reversible, we provide a necessary and sufficient condition for all fluctuation spectra to be monotonic on [0,π] and (ii) if it is irreversible, then there exist some nonmonotonic fluctuation spectra.
General Minimal Diffusion Process: its Construction, Invariant Measure, Entropy Production and Irreversibility

Da-Quan Jiang,Min Qian,Min-Ping Qian

DOI: https://doi.org/10.1007/978-3-540-40957-1_4

2004-01-01

Abstract: • 3.1 Construction of the Minimal Semigroup 3.1.1 Resolvent Operators and Minimal Semigroup in a Bounded Domain 3.1.2 Resolvent Operators 3.1.3 Banach Space and the Minimal Semigroup • 3.2 Ergodic Stationary Diffusion Process 3.2.1 Transition Functions 3.2.2 Invariant Functional and Invariant Measure 3.2.3 Uniqueness of Invariant Probability Measure 3.2.4 Foguel Alternatives and Ergodicity • 3.3 Irreversibility and Entropy Production 3.3.1 Heuristic Derivation for Entropy Production Rate 3.3.2 Reversibility, Symmetry and Zero Entropy Production
Full $Γ$-expansion of reversible Markov chains level two large deviations rate functionals

Claudio Landim,Ricardo Misturini,Federico Sau

2024-07-01

Abstract:Let $\Xi_n \subset \mathbb R^d$, $n\ge 1$, be a sequence of finite sets and consider a $\Xi_n$-valued, irreducible, reversible, continuous-time Markov chain $(X^{(n)}_t:t\ge 0)$. Denote by $\mathscr P(\mathbb R^d) $ the set of probability measures on $\mathbb R^d$ and by $I_n\colon \mathscr P(\mathbb R^d) \to [0,+\infty)$ the level two large deviations rate functional for $X^{(n)}_t$ as $t\to\infty$. We present a general method, based on tools used to prove the metastable behaviour of Markov chains, to derive a full expansion of $I_n$ expressing it as $I_n = I^{(0)} \,+\, \sum_{1\le p\le q} (1/\theta^{(p)}_n)\, I^{(p)}$, where $I^{(p)}\colon \mathscr P(\mathbb R^d) \to [0,+\infty]$ represent rate functionals independent of $n$ and $\theta^{(p)}_n$ sequences such that $\theta^{(1)}_n \to\infty$, $\theta^{(p)}_n / \theta^{(p+1)}_n \to 0$ for $1\le p< q$. The speed $\theta^{(p)}_n$ corresponds to the time-scale at which the Markov chains $X^{(n)}_t$ exhibits a metastable behavior, and the $I^{(p-1)}$ zero-level sets to the metastable states. To illustrate the theory we apply the method to random walks in potential fields.

Probability,Statistical Mechanics,Analysis of PDEs
Stationary Distributions of Continuous-Time Markov Chains: A Review of Theory and Truncation-Based Approximations

Juan Kuntz,Philipp Thomas,Guy-Bart Stan,Mauricio Barahona

DOI: https://doi.org/10.1137/19M1289625

IF: 10.2

2021-02-06

SIAM Review

Abstract:SIAM Review, Volume 63, Issue 1 , Page 3-64, January 2021. Computing the stationary distributions of a continuous-time Markov chain (CTMC) involves solving a set of linear equations. In most cases of interest, the number of equations is infinite or too large, and the equations cannot be solved analytically or numerically. Several approximation schemes overcome this issue by truncating the state space to a manageable size. In this review, we first give a comprehensive theoretical account of the stationary distributions and their relation to the long-term behaviour of CTMCs that is readily accessible to non-experts and free of irreducibility assumptions made in standard texts. We then review truncation-based approximation schemes for CTMCs with infinite state spaces paying particular attention to the schemes' convergence and the errors they introduce, and we illustrate their performance with an example of a stochastic reaction network of relevance in biology and chemistry. We conclude by elaborating on computational trade-offs associated with error control and several open questions.

mathematics, applied

2 Circulation Distribution, Entropy Production and Irreversibility of Finite Markov Chains with Continuous Parameter

Circulation Distribution, Entropy Production And Irreversibility Of Denumerable Markov Chains

Thermodynamics of Markov Processes with Non-extensive Entropy and Free Energy

Circulations of Markov-Chains with Continuous-Time and the Probability Interpretation of Some Determinants

Cycle Symmetries and Circulation Fluctuations for Discrete-Time and Continuous-Time Markov Chains

5 Entropy Production, Rotation Numbers and Irreversibility of Diffusion Processes on Manifolds

Reversibility and Entropy Production of Inhomogeneous Markov Chains

A Simple Discrete Model of Brownian Motors: Time-periodic Markov Chains

Large deviations and related problems for absorbing Markov chains

The Green–Kubo formula, autocorrelation function and fluctuation spectrum for finite Markov chains with continuous time

Advanced method and application on Markov state transfer chain

The invariant meauures, probability flux and circulations of one-dimensional Markov processes

Irreversible Circulation of Fluctuation and Entropy Production

Canonical structure and orthogonality of forces and currents in irreversible Markov chains

Large Deviations for Markov Chains and Spectrum of Matrices

Representation and Characterization of Quasistationary Distributions for Markov Chains

Non-equilibrium Stochastic Dynamics

On the Relation Between Reversibility and Monotonicity of Fluctuation Spectra for Discrete Time Finite State Markov Chains

General Minimal Diffusion Process: its Construction, Invariant Measure, Entropy Production and Irreversibility

Full $Γ$-expansion of reversible Markov chains level two large deviations rate functionals

Stationary Distributions of Continuous-Time Markov Chains: A Review of Theory and Truncation-Based Approximations