Abstract:Co-occurrence statistics for sequential data are common and important data signals in machine learning, which provide rich correlation and clustering information about the underlying object space. We give the first bound on the convergence rate of estimating the co-occurrence matrix of a regular (aperiodic and irreducible) finite Markov chain from a single random trajectory. Our work is motivated by the analysis of a well-known graph learning algorithm DeepWalk by [Qiu et al. WSDM '18], who study the convergence (in probability) of co-occurrence matrix from random walk on undirected graphs in the limit, but left the convergence rate an open problem. We prove a Chernoff-type bound for sums of matrix-valued random variables sampled via an ergodic Markov chain, generalizing the regular undirected graph case studied by [Garg et al. STOC '18]. Using the Chernoff-type bound, we show that given a regular Markov chain with $n$ states and mixing time $\tau$, we need a trajectory of length $O(\tau (\log{(n)}+\log{(\tau)})/\epsilon^2)$ to achieve an estimator of the co-occurrence matrix with error bound $\epsilon$. We conduct several experiments and the experimental results are consistent with the exponentially fast convergence rate from theoretical analysis. Our result gives the first sample complexity analysis in graph representation learning.

Blockwise perturbation theory for nearly uncoupled Markov chains and its application

Blockwise Perturbation Theory for Markov Chains

Blockwise Perturbation Theory for Block P-Cyclic Stochastic Matrices

A note on entrywise perturbation theory for Markov chains

Entrywise perturbation theory for rate matrices of GI/M/1 type Markov chains

Perturbation Analysis in Verification of Discrete-Time Markov Chains.

Metastability of Exponentially Perturbed Markov Chains

Perturbations of Markov Chains

Singular value distribution of dense random matrices with block Markovian dependence

Probabilistic perturbation bounds of matrix decompositions

Perturbation theory for killed Markov processes and quasi-stationary distributions

Error bounds for last-column-block-augmented truncations of block-structured Markov chains

A Matrix Chernoff Bound for Markov Chains and Its Application to Co-Occurrence Matrices

An introduction to the Markov chain Monte Carlo method

Concentration Bounds for Co-occurrence Matrices of Markov Chains.

Markov chains: Structure and applications

Clustering in Block Markov Chains

Estimating the number of clusters of a Block Markov Chain

Graph-combinatorial approach for large deviations of Markov chains

Markov chains and mappings of distributions on compact spaces II: Numerics and Conjectures

An effective introduction to the Markov Chain Monte Carlo method