Condenser physics applied to Markov chains - A brief introduction to potential theory

A. Gaudilliere

DOI: https://doi.org/10.48550/arXiv.0901.3053

2009-01-20

Abstract:These notes constitute the introduction to potential theory I exposed at the XIIth brazilian school of probability inside Elisabetta Scoppola's Introduction to Metastability.

Probability,Mathematical Physics

What problem does this paper attempt to address?

Stochastic Analysis of Dynamical System with Double-Well Potential

W. Q. Zhu,G. Q. Cai,R. C. Hu

DOI: https://doi.org/10.1007/s40435-013-0002-4

2013-01-01

International Journal of Dynamics and Control

Abstract:The method of quasi-conservative averaging, or the stochastic averaging of energy envelope, is extended to a system with a double-well potential subjected to both the external and parametric non-white noise excitations. Instead of using the power spectral densities, the correlation functions of the excitation processes are used directly so that the averaging procedure can be performed for different types of motions appearing in the system with a double-well potential. The obtained Markov process of the energy envelope allows to carry out the dynamic analysis of the system, such as to assess the asymptotic behaviors at the boundaries, to find the probability density of the response, and to investigate the transition of the system motion between the two wells, as done in the present paper. Furthermore, the proposed procedure can be applied not only to systems with double-well potentials, but also to those with potentials of more complicated shapes.
Some Probabilistic Properties of General Topological Markov Chains

Eduardo A. Silva,Elis G. Mesquita,Edgar Matias

2023-10-20

Abstract:In this paper, we utilize the framework of Markov processes to attain a more probabilistic perspective on the theory of transfer operators. In doing so, we establish a functional central limit theorem (FLCT) for an $O(N)$ model associated with Dyson potential on the one-dimensional lattice. We also proof a FLCT on a non-compact alphabet setting, for a model associated with a Dyson type potential on the one-dimensional lattice. A Breimann ergodic theorem for equilibrium measures arising from the Ruelle Operator Formalism is proved. Furthermore, we obtain a qualitative criterion (strong transitivity) to determine when the conformal measure has full support. As an application, we show how to connect the Ruelle operator framework with the perspective of the Hopf theory of Markov Processes.

Dynamical Systems
Potential Theory of Normal Tempered Stable Process

Arun Kumar,Harsh Verma

DOI: https://doi.org/10.48550/arXiv.2004.02267

2020-04-06

Abstract:In this article, we study the potential theory of normal tempered stable process which is obtained by time-changing the Brownian motion with a tempered stable subordinator. Precisely, we study the asymptotic behavior of potential density and Levy density associated with tempered stable subordinator and the Green function and the Levy density associated with the normal tempered stable process. We also provide the corresponding results for normal inverse Gaussian process which is a well studied process in literature.

Probability
Stable limits for Markov chains via the Principle of Conditioning

Mohamed El Machkouri,Adam Jakubowski,Dalibor Volný

2018-08-14

Abstract:We study limit theorems for partial sums of instantaneous functions of a homogeneous Markov chain on a general state space. The summands are heavy-tailed and the limits are stable distributions. The conditions imposed on the transition operator $P$ of the Markov chain ensure that the limit is the same as if the summands were independent. Such a~scheme admits a physical interpretation, as given in Jara et al. (Ann. Appl. Probab., 19 (2009), 2270--2300).

Mathematics
An introduction to the Markov chain Monte Carlo method

Wenlong Wang

DOI: https://doi.org/10.1119/5.0122488

IF: 0.835

2022-12-02

American Journal of Physics

Abstract:We present an intuitive, conceptual, and semi-rigorous introduction to the Markov Chain Monte Carlo method using a simple model of population dynamics and focusing on a few elementary distributions. We start from two states, then three states, and finally generalize to many states with both discrete and continuous distributions. Despite the mathematical simplicity, our examples include the essential concepts of the Markov Chain Monte Carlo method, including ergodicity, global balance and detailed balance, proposal or selection probability, acceptance probability, the underlying stochastic matrix, and error analysis. Our experience suggests that most senior undergraduate students in physics can follow these materials without much difficulty.
The noise intensity of a Markov chain

Lukas Ramlow,Benjamin Lindner

2024-02-16

Abstract:Stochastic transitions between discrete microscopic states play an important role in many physical and biological systems. Often, these transitions lead to fluctuations on a macroscopic scale. A classic example from neuroscience is the stochastic opening and closing of ion channels and the resulting fluctuations in membrane current. When the microscopic transitions are fast, the macroscopic fluctuations are nearly uncorrelated and can be fully characterized by their mean and noise intensity. We show how, for an arbitrary Markov chain, the noise intensity can be determined from an algebraic equation, based on the transition rate matrix. We demonstrate the validity of the theory using an analytically tractable two-state Markovian dichotomous noise, an eight-state model for a Calcium channel subunit (De Young-Keizer model), and Markov models of the voltage-gated Sodium and Potassium channels as they appear in a stochastic version of the Hodgkin-Huxley model.

Statistical Mechanics,Data Analysis, Statistics and Probability
Compressibility and Stochastic Stability of Monotone Markov Chain

Sergey Foss,Michael Scheutzow

2024-03-22

Abstract:For a stochastically monotone Markov chain taking values in a Polish space, we present a number of conditions for existence and for uniqueness of its stationary regime, as well as for closeness of its transient trajectories. In particular, we generalise a basic result by Bhattacharya and Majumdar (2007) where a certain form of mixing, or swap condition was assumed uniformly over the state space. We do not rely on continuity properties of transition probabilities.

Probability
Noise intensity of a Markov chain

Lukas Ramlow,Benjamin Lindner

DOI: https://doi.org/10.1103/physreve.110.014139

IF: 2.707

2024-07-26

Physical Review E

Abstract:Stochastic transitions between discrete microscopic states play an important role in many physical and biological systems. Often these transitions lead to fluctuations on a macroscopic scale. A classic example from neuroscience is the stochastic opening and closing of ion channels and the resulting fluctuations in membrane current. When the microscopic transitions are fast, the macroscopic fluctuations are nearly uncorrelated and can be fully characterized by their mean and noise intensity. We show how, for an arbitrary Markov chain, the noise intensity can be determined from an algebraic equation, based on the transition rate matrix; these results are in agreement with earlier results from the theory of zero-frequency noise in quantum mechanical and classical systems. We demonstrate the validity of the theory using an analytically tractable two-state Markovian dichotomous noise, an eight-state model for a calcium channel subunit (De Young-Keizer model), and Markov models of the voltage-gated sodium and potassium channels as they appear in a stochastic version of the Hodgkin-Huxley model. https://doi.org/10.1103/PhysRevE.110.014139 ©2024 American Physical Society

physics, fluids & plasmas, mathematical
Living on the edge of instability

Artem Ryabov,Viktor Holubec,Ekaterina Berestneva

DOI: https://doi.org/10.1088/1742-5468/ab333f

2019-08-12

Abstract:Statistical description of stochastic dynamics in highly unstable potentials is strongly affected by properties of divergent trajectories, that quickly leave meta-stable regions of the potential landscape and never return. Using ideas from theory of Q-processes and quasi-stationary distributions, we analyze position statistics of non-diverging trajectories. We discuss two limit distributions which can be considered as (formal) generalizations of the Gibbs canonical distribution to highly unstable systems. Even though the associated effective potentials differ only slightly, properties of the two distributions are fundamentally different for all highly unstable system. The distribution for trajectories conditioned to diverge in an infinitely distant future is localized and light-tailed. The other distribution, describing trajectories surviving in the meta-stable region at the instant of conditioning, is heavy-tailed. The exponent of the corresponding power-law tail is determined by the leading divergent term of the unstable potential. We discuss different equivalent forms of the two distributions and derive properties of the effective statistical force arising in the ensemble of non-diverging trajectories after the Doob h-transform. The obtained explicit results generically apply to non-linear dynamical models with meta-stable states and fast kinetic transitions.

Probability
Introduction to the theory of stochastic processes and Brownian motion problems

J. L. Garcia-Palacios

DOI: https://doi.org/10.48550/arXiv.cond-mat/0701242

2007-01-11

Abstract:These notes are an introduction to the theory of stochastic processes based on several sources. The presentation mainly follows the books of van Kampen and Wio, except for the introduction, which is taken from the book of Gardiner and the parts devoted to the Langevin equation and the methods for solving Langevin and Fokker-Planck equations, which are based on the book of Risken.

Statistical Mechanics,Data Analysis, Statistics and Probability
On the concentration properties of Interacting particle processes

Pierre Del Moral,Peng Hu,Liming Wu

DOI: https://doi.org/10.48550/arXiv.1107.1948

2011-07-12

Abstract:These lecture notes present some new concentration inequalities for Feynman-Kac particle processes. We analyze different types of stochastic particle models, including particle profile occupation measures, genealogical tree based evolution models, particle free energies, as well as backward Markov chain particle models. We illustrate these results with a series of topics related to computational physics and biology, stochastic optimization, signal processing and bayesian statistics, and many other probabilistic machine learning algorithms. Special emphasis is given to the stochastic modeling and the quantitative performance analysis of a series of advanced Monte Carlo methods, including particle filters, genetic type island models, Markov bridge models, interacting particle Markov chain Monte Carlo methodologies.

Numerical Analysis,Probability
Probability Thermodynamics and Probability Quantum Field

Ping Zhang,Wen-Du Li,Tong Liu,Wu-Sheng Dai

DOI: https://doi.org/10.1063/5.0150370

2023-10-24

Abstract:We introduce probability thermodynamics and probability quantum fields. By probability we mean that there is an unknown operator, physical or nonphysical, whose eigenvalues obey a certain statistical distribution. Eigenvalue spectra define spectral functions. Various thermodynamic quantities in thermodynamics and effective actions in quantum field theory are all spectral functions. In the scheme, eigenvalues obey a probability distribution, so a probability distribution determines a family of spectral functions in thermodynamics and quantum field theory. This leads to probability thermodynamics and probability quantum fields determined by a probability distribution. In constructing spectral functions, we encounter a problem. The conventional definition of spectral functions applies only to lower bounded spectra. In our scheme, however, there are two types of spectra: lower bounded spectra, corresponding to the probability distribution with nonnegative random variables, and the lower unbounded spectra, corresponding to probability distributions with negative random variables. To take the lower unbounded spectra into account, we generalize the definition of spectral functions by analytical continuation. In some cases, we encounter divergences. We remove the divergence by a renormalization procedure. Moreover, in virtue of spectral theory in physics, we generalize some concepts in probability theory. For example, the moment-generating function in probability theory does not always exist. We redefine the moment-generating function as the generalized heat kernel introduced in this paper, which makes the concept definable when the definition in probability theory fails. Thermodynamic quantities, vacuum amplitudes, one-loop effective actions, and vacuum energies for various probability distributions are presented.

Statistical Mechanics
Introduction to the field theory of classical and quantum phase transitions

Flavio S. Nogueira

DOI: https://doi.org/10.48550/arXiv.1009.1603

2010-09-09

Abstract:These lecture notes provide a relatively self-contained introduction to field theoretic methods employed in the study of classical and quantum phase transitions.

Statistical Mechanics,High Energy Physics - Theory
The probabilistic world

C. Wetterich

2023-09-26

Abstract:Physics is based on probabilities as fundamental entities of a mathematical description. Expectation values of observables are computed according to the classical statistical rule. The overall probability distribution for one world covers all times. The quantum formalism arises once one focuses on the transport of the time-local probabilistic information from one hypersurface to a neighboring one. Wave functions or the density matrix allow the formulation of a general linear evolution law for classical statistics. The density matrix for classical statistics is a powerful tool which allows us to implement for generalized Ising models concepts as basis transformations, the momentum observable and the associated Fourier representation, or the definition of subsystems by subtraces of the density matrix. The association of operators to observables permits the computation of expectation values in terms of the density matrix by the usual quantum rule. We show that probabilistic cellular automata are quantum systems in a formulation with discrete time steps and real wave functions. The evolution operator for automata can be expressed in terms of fermionic creation and annihilation operators. The time-local probabilistic information amounts to a subsystem of the overall probabilistic system which is correlated with its environment consisting of the past and future. Such subsystems typically involve probabilistic observables for which only a probability distribution for their possible measurement values is available. Incomplete statistics does not permit to compute classical correlation functions for arbitrary subsystem-observables since different overall observables are mapped to the same subsystem observable. Bell's inequalities are not generally applicable.

Quantum Physics,Disordered Systems and Neural Networks,Statistical Mechanics,High Energy Physics - Theory
Markov Chains Approximations of jump-Diffusion Quantum Trajectories

Clement Pellegrini

DOI: https://doi.org/10.48550/arXiv.0803.2593

2008-03-18

Abstract:"Quantum trajectories" are solutions of stochastic differential equations also called Belavkin or Stochastic Schrödinger Equations. They describe random phenomena in quantum measurement theory. Two types of such equations are usually considered, one is driven by a one-dimensional Brownian motion and the other is driven by a counting process. In this article, we present a way to obtain more advanced models which use jump-diffusion stochastic differential equations. Such models come from solutions of martingale problems for infinitesimal generators. These generators are obtained from the limit of generators of classical Markov chains which describe discrete models of quantum trajectories. Furthermore, stochastic models of jump-diffusion equations are physically justified by proving that their solutions can be obtained as the limit of the discrete trajectories.

Probability
A tale of three approaches: dynamical phase transitions for weakly bound Brownian particles

Lucianno Defaveri,Eli Barkai,David A. Kessler

2024-07-24

Abstract:We investigate a system of Brownian particles weakly bound by attractive parity-symmetric potentials that grow at large distances as $V(x) \sim |x|^\alpha$, with $0 < \alpha < 1$. The probability density function $P(x,t)$ at long times reaches the Boltzmann-Gibbs equilibrium state, with all moments finite. However, the system's relaxation is not exponential, as is usual for a confining system with a well-defined equilibrium, but instead follows a stretched exponential $e^{- \mathrm{const} \, t^\nu}$ with exponent $\nu=\alpha/(2+\alpha)$. This problem is studied from three perspectives. First, we propose a straightforward and general scaling rate-function solution for $P(x,t)$. This rate-function, which is an important tool from large deviation theory, also displays anomalous time scaling and a dynamical phase transition. Second, through the eigenfunctions of the Fokker-Planck operator, we obtain, using the WKB method, more complete solutions that reproduce the rate function approach. Finally, we show how the alternative path-integral formalism allows us to recover the same results, with the above rate-function being the solution of the classical Hamilton-Jacobi equation describing the most probable path. Properties such as parity, the role of initial conditions, and the dynamical phase transition are thoroughly studied in all three approaches.

Statistical Mechanics
Entropy and variational principle for one-dimensional lattice systems with a general a priori probability: positive and zero temperature

A. O. LOPES,J. K. MENGUE,J. MOHR,R. R. SOUZA

DOI: https://doi.org/10.1017/etds.2014.15

2014-07-03

Ergodic Theory and Dynamical Systems

Abstract:We generalize several results of the classical theory of thermodynamic formalism by considering a compact metric space $M$ as the state space. We analyze the shift acting on $M^{\mathbb{N}}$ and consider a general a priori probability for defining the transfer (Ruelle) operator. We study potentials $A$ which can depend on the infinite set of coordinates in $M^{\mathbb{N}}$ . We define entropy and by its very nature it is always a non-positive number. The concepts of entropy and transfer operator are linked. If $M$ is not a finite set there exist Gibbs states with arbitrary negative value of entropy. Invariant probabilities with support in a fixed point will have entropy equal to minus infinity. In the case $M=S^{1}$ , and the a priori measure is Lebesgue $dx$ , the infinite product of $dx$ on $(S^{1})^{\mathbb{N}}$ will have zero entropy. We analyze the Pressure problem for a Hölder potential $A$ and its relation with eigenfunctions and eigenprobabilities of the Ruelle operator. Among other things we analyze the case where temperature goes to zero and we show some selection results. Our general setting can be adapted in order to analyze the thermodynamic formalism for the Bernoulli space with countable infinite symbols. Moreover, the so-called $XY$ model also fits under our setting. In this last case $M$ is the unitary circle $S^{1}$ . We explore the differentiable structure of $(S^{1})^{\mathbb{N}}$ by considering a certain class of smooth potentials and we show some properties of the corresponding main eigenfunctions.

mathematics, applied
Markov Chain Monte Carlo Methods in Quantum Field Theories: A Modern Primer

Anosh Joseph

DOI: https://doi.org/10.48550/arXiv.1912.10997

2019-12-23

High Energy Physics - Theory

Abstract:We introduce and discuss Monte Carlo methods in quantum field theories. Methods of independent Monte Carlo, such as random sampling and importance sampling, and methods of dependent Monte Carlo, such as Metropolis sampling and Hamiltonian Monte Carlo, are introduced. We review the underlying theoretical foundations of Markov chain Monte Carlo. We provide several examples of Monte Carlo simulations, including one-dimensional simple harmonic oscillator, unitary matrix model exhibiting Gross-Witten-Wadia transition and a supersymmetric model exhibiting dynamical supersymmetry breaking.
Markov Chains and Kinetic Theory: A Possible Application to Socio-Economic Problems

Bruno Carbonaro,Marco Menale

DOI: https://doi.org/10.3390/math12101571

IF: 2.4

2024-05-18

Mathematics

Abstract:A very important class of models widely used nowadays to describe and predict, at least in stochastic terms, the behavior of many-particle systems (where the word "particle" is not meant in the purely mechanical sense: particles can be cells of a living tissue, or cars in a traffic flow, or even members of an animal or human population) is the Kinetic Theory for Active Particles, i.e., a scheme of possible generalizations and re-interpretations of the Boltzmann equation. Now, though in the literature on the subject this point is systematically disregarded, this scheme is based on Markov Chains, which are special stochastic processes with important properties they share with many natural processes. This circumstance is here carefully discussed not only to suggest the different ways in which Markov Chains can intervene in equations describing the stochastic behavior of any many-particle system, but also, as a preliminary methodological step, to point out the way in which the notion of a Markov Chain can be suitably generalized to this aim. As a final result of the discussion, we find how to develop new very plausible and likely ways to take into account possible effects of the external world on a non-isolated many-particle system, with particular attention paid to socio-economic problems.

mathematics
Stochastic Thermodynamics of Brownian motion in Temperature Gradient

Mingnan Ding,Jun Wu,Xiangjun Xing

2024-02-21

Abstract:We study stochastic thermodynamics of a Brownian particle which is subjected to a temperature gradient and is confined by an external potential. We first formulate an over-damped Ito-Langevin theory in terms of local temperature, friction coefficient, and steady state distribution, all of which are experimentally measurable. We then study the associated stochastic thermodynamics theory. We analyze the excess entropy production (EP) both at trajectory level and at ensemble level, and derive the Clausius inequality as well as the transient fluctuation theorem (FT). We also use molecular dynamics to simulate a Brownian particle inside a Lennard-Jones fluid and verify the FT. Remarkably we find that the FT remains valid even in the under-damped regime. We explain the possible mechanism underlying this surprising result.

Statistical Mechanics,Soft Condensed Matter

Condenser physics applied to Markov chains - A brief introduction to potential theory

Stochastic Analysis of Dynamical System with Double-Well Potential

Some Probabilistic Properties of General Topological Markov Chains

Potential Theory of Normal Tempered Stable Process

Stable limits for Markov chains via the Principle of Conditioning

An introduction to the Markov chain Monte Carlo method

The noise intensity of a Markov chain

Compressibility and Stochastic Stability of Monotone Markov Chain

Noise intensity of a Markov chain

Living on the edge of instability

Introduction to the theory of stochastic processes and Brownian motion problems

On the concentration properties of Interacting particle processes

Probability Thermodynamics and Probability Quantum Field

Introduction to the field theory of classical and quantum phase transitions

The probabilistic world

Markov Chains Approximations of jump-Diffusion Quantum Trajectories

A tale of three approaches: dynamical phase transitions for weakly bound Brownian particles

Entropy and variational principle for one-dimensional lattice systems with a general a priori probability: positive and zero temperature

Markov Chain Monte Carlo Methods in Quantum Field Theories: A Modern Primer

Markov Chains and Kinetic Theory: A Possible Application to Socio-Economic Problems

Stochastic Thermodynamics of Brownian motion in Temperature Gradient