Abstract:Many existing results on statistical learning theory are based on the assumption that samples are independently and identically distributed (i.i.d.). However, the assumption of i.i.d. samples is not suitable for practical application to problems in which samples are time dependent. In this paper, we are mainly concerned with the empirical risk minimization (ERM) based learning process for time-dependent samples drawn from a continuous-time Markov chain. This learning process covers many kinds of practical applications, e.g., the prediction for a time series and the estimation of channel state information. Thus, it is significant to study its theoretical properties including the generalization bound, the asymptotic convergence, and the rate of convergence. It is noteworthy that, since samples are time dependent in this learning process, the concerns of this paper cannot (at least straightforwardly) be addressed by existing methods developed under the sample i.i.d. assumption. We first develop a deviation inequality for a sequence of time-dependent samples drawn from a continuous-time Markov chain and present a symmetrization inequality for such a sequence. By using the resultant deviation inequality and symmetrization inequality, we then obtain the generalization bounds of the ERM-based learning process for time-dependent samples drawn from a continuous-time Markov chain. Finally, based on the resultant generalization bounds, we analyze the asymptotic convergence and the rate of convergence of the learning process.

Generalization Bound for Infinitely Divisible Empirical Process.

Risk Bounds for Infinitely Divisible Distribution

LIMIT BEHAVIORS FOR THE INCREMENTS OF A D-Dimensional MULTI-PARAMETER GAUSSIAN PROCESS

Limit Theory for A General Class of Garch Models with Just Barely Infinite Variance

On the Increments of A D-Dimensional Gaussian Process

Limit theorems for a class of processes generalizing the U-empirical process

On Large Increments of Infinite Series of Ornstein-Uhlenbeck Processes

On large increments and liminfs of a two-parameter Gaussian process

Generalization Bounds of Erm Algorithm with V-Geometrically Ergodic Markov Chains

Some Limit Theorems for Linear Processes Generated by Symmetrically Exchangeable Random Variables

Extremal theory for long range dependent infinitely divisible processes

Large Deviations for Empirical Processes of Interacting Particle Systems

A General Law of Moment Convergence Rates for Uniform Empirical Process

Deviation inequalities for contractive infinite memory process

Generalization Bounds of ERM-based Learning Processes for Continuous-Time Markov Chains.

A LIL for Independent Non-Identically Distributed Random Variables in Banach Space and Its Applications

Generalization bounds for nonparametric regression with $β-$mixing samples

Generalization Bounds of ERM Algorithm with Markov Chain Samples

On Generalized Sub-Gaussian Canonical Processes and Their Applications

Large Deviations for Infinite Dimensional and Reversible Reaction-Diffusion Processes

Large deviation for Gibbs probabilities at zero temperature and invariant idempotent probabilities for iterated function systems