Abstract:Machine learning algorithms have been applied to predict agent behaviors in real-world dynamic systems, such as advertiser behaviors in sponsored search and worker behaviors in crowdsourcing. The behavior data in these systems are generated by live agents: once the systems change due to the adoption of the prediction models learnt from the behavior data, agents will observe and respond to these changes by changing their own behaviors accordingly. As a result, the behavior data will evolve and will not be identically and independently distributed, posing great challenges to the theoretical analysis on the machine learning algorithms for behavior prediction. To tackle this challenge, in this paper, we propose to use Markov Chain in Random Environments (MCRE) to describe the behavior data, and perform generalization analysis of the machine learning algorithms on its basis. Since the one-step transition probability matrix of MCRE depends on both previous states and the random environment, conventional techniques for generalization analysis cannot be directly applied. To address this issue, we propose a novel technique that transforms the original MCRE into a higher-dimensional time-homogeneous Markov chain. The new Markov chain involves more variables but is more regular, and thus easier to deal with. We prove the convergence of the new Markov chain when time approaches infinity. Then we prove a generalization bound for the machine learning algorithms on the behavior data generated by the new Markov chain, which depends on both the Markovian parameters and the covering number of the function class compounded by the loss function for behavior prediction and the behavior prediction model. To the best of our knowledge, this is the first work that performs the generalization analysis on data generated by complex processes in real-world dynamic systems.

Learning from uniformly ergodic Markov chains

Generalization Bounds of ERM-based Learning Processes for Continuous-Time Markov Chains.

Geometric ergodicity of trans-dimensional Markov chain Monte Carlo algorithms

Learning performance of regularized regression with multiscale kernels based on Markov observations

Universal Rates of Empirical Risk Minimization

The Generalized Entropy Ergodic Theorem for Nonhomogeneous Markov Chains

ERM Learning with Unbounded Sampling

Fast Rates of ERM and Stochastic Approximation: Adaptive to Error Bound Conditions

Lower bounds on the rate of convergence for accept-reject-based Markov chains in Wasserstein and total variation distances

The Theory and Algorithm of Ergodic Inference

Upper and lower bounds on the subgeometric convergence of adaptive Markov chain Monte Carlo

On the Tightness of Information-Theoretic Bounds on Generalization Error of Learning Algorithms.

An invitation to adaptive Markov chain Monte Carlo convergence theory

Agent Behavior Prediction and Its Generalization Analysis

Error bounds for computing the expectation by Markov chain Monte Carlo

Hoeffding's lemma for Markov Chains and its applications to statistical learning

Generalization Capacity of Multi-class SVM Based on Markovian Resampling

Generalization Error Bounds for Optimization Algorithms Via Stability

Equivalences of Geometric Ergodicity of Markov Chains

Hoeffding's Inequality for Markov Chains under Generalized Concentrability Condition

Error Density-dependent Empirical Risk Minimization