Abstract:A rigorous and largely self-contained account of (a) the bread-and-butter concepts and techniques in Markov chain theory and (b) the long-term behaviour of chains. As much as possible, the treatment is probabilistic instead of analytical (I stay away from semigroup theory). Personally, I tend to find that the intuition lies with the former and not the latter. This manuscript is geared towards those interested in the use of Markov chains as models of real-life phenomena. For this reason, I focus on the type of chains most commonly encountered in practice (time-homogeneous, minimal, and right-continuous in the discrete topology) and choose a starting point very familiar to this audience: the (Kendall-) Gillespie Algorithm commonly used to simulate these chains. In order to keep the prerequisite knowledge and technical complications to a minimum, I take a 'bare-bones' approach that keeps the focus on chains (instead of more general processes) to an almost pathological degree: I use the 'jump and hold' structure of chains extensively; almost no martingale theory; minimal coupling; no stochastic calculus; and, even though regeneration and renewal (of course!) feature in the manuscript, they do so exclusively in the context of chains. I have also taken some extra steps to avoid imposing certain assumptions encountered in other texts that sometimes prove to be stumbling blocks in practice (e.g., irreducibility of the state space, boundedness of test functions and of stopping times). For more details, see the preface.

An Alternative Markov Property for Chain Graphs

Learning AMP Chain Graphs under Faithfulness

Reversible Mcmc on Markov Equivalence Classes of Sparse Directed Acyclic Graphs

Markov Equivalence Classes for Maximal Ancestral Graphs

Chain Graphs for Learning

Markov chains revisited

Pairwise Markov properties for regression graphs

Identifiability and Consistent Estimation for Gaussian Chain Graph Models

Geometry and Duality of Alternating Markov Chains

Probabilistic image modeling with an extended chain graph for human activity recognition and image segmentation.

Longitudinal Network Models and Permutation-Uniform Markov Chains

Markov tail chains

Bayesian MAP Model Selection of Chain Event Graphs

Distributional Invariances and Interventional Markov Equivalence for Mixed Graph Models

Structural Learning Of Chain Graphs Via Decomposition

Characterization of causal ancestral graphs for time series with latent confounders

Non-existence of Markovian time dynamics for graphical models of correlated default

Graphs: Associated Markov Chains

Conditional Adjustment in a Markov Equivalence Class

Time-varying Graph Learning Under Structured Temporal Priors

A new method for tackling asymmetric decision problems