Abstract:Empirical process theory for i.i.d. observations has emerged as a ubiquitous tool for understanding the generalization properties of various statistical problems. However, in many applications where the data exhibit temporal dependencies (e.g., in finance, medical imaging, weather forecasting etc.), the corresponding empirical processes are much less understood. Motivated by this observation, we present a general bound on the expected supremum of empirical processes under standard $\beta/\rho$-mixing assumptions. Unlike most prior work, our results cover both the long and the short-range regimes of dependence. Our main result shows that a non-trivial trade-off between the complexity of the underlying function class and the dependence among the observations characterizes the learning rate in a large class of nonparametric problems. This trade-off reveals a new phenomenon, namely that even under long-range dependence, it is possible to attain the same rates as in the i.i.d. setting, provided the underlying function class is complex enough. We demonstrate the practical implications of our findings by analyzing various statistical estimators in both fixed and growing dimensions. Our main examples include a comprehensive case study of generalization error bounds in nonparametric regression over smoothness classes in fixed as well as growing dimension using neural nets, shape-restricted multivariate convex regression, estimating the optimal transport (Wasserstein) distance between two probability distributions, and classification under the Mammen-Tsybakov margin condition -- all under appropriate mixing assumptions. In the process, we also develop bounds on $L_r$ ($1\le r\le 2$)-localized empirical processes with dependent observations, which we then leverage to get faster rates for (a) tuning-free adaptation, and (b) set-structured learning problems.

Learning Generalization and Regularization of Nonhomogeneous Temporal Poisson Processes

Learning general temporal point processes based on dynamic weight generation

Learning granger causality for non-stationary Hawkes processes

Comparative Study of the Non-Homogeneous Poisson Process Type-I Generalized Half-Logistic Distribution

Compound Poisson Processes, Latent Shrinkage Priors and Bayesian Nonconvex Penalization

Learning Conditional Generative Models for Temporal Point Processes

Cumulative Hazard Function Based Efficient Multivariate Temporal Point Process Learning

Trade-off Between Dependence and Complexity for Nonparametric Learning -- an Empirical Process Approach

Negative-Binomial Randomized Gamma Markov Processes for Heterogeneous Overdispersed Count Time Series

Meta-Learning for Neural Network-based Temporal Point Processes

Probabilistic Learning of Multivariate Time Series with Temporal Irregularity

Fully Neural Network based Model for General Temporal Point Processes

Non-homogeneous Poisson and linear regression models as approaches to study time series with change-points

Bridging the Gap: A Generalized Stochastic Process for Count Data

Bayesian Geostatistical Modeling for Discrete-Valued Processes

Nonlinear Hawkes Processes in Time-Varying System

Strongly convergent homogeneous approximations to inhomogeneous Markov jump processes and applications

Bayesian inference for Matérn repulsive processes

A non-homogeneous count process: marginalizing a Poisson driven Cox process

On Non-asymptotic Theory of Recurrent Neural Networks in Temporal Point Processes