Abstract:In this paper, we study a class of deterministically constrained stochastic optimization problems. Existing methods typically aim to find an $\epsilon$-stochastic stationary point, where the expected violations of both constraints and first-order stationarity are within a prescribed accuracy $\epsilon$. However, in many practical applications, it is crucial that the constraints be nearly satisfied with certainty, making such an $\epsilon$-stochastic stationary point potentially undesirable due to the risk of significant constraint violations. To address this issue, we propose single-loop variance-reduced stochastic first-order methods, where the stochastic gradient of the stochastic component is computed using either a truncated recursive momentum scheme or a truncated Polyak momentum scheme for variance reduction, while the gradient of the deterministic component is computed exactly. Under the error bound condition with a parameter $\theta \geq 1$ and other suitable assumptions, we establish that these methods respectively achieve a sample and first-order operation complexity of $\widetilde O(\epsilon^{-\max\{\theta+2, 2\theta\}})$ and $\widetilde O(\epsilon^{-\max\{4, 2\theta\}})$ for finding a stronger $\epsilon$-stochastic stationary point, where the constraint violation is within $\epsilon$ with certainty, and the expected violation of first-order stationarity is within $\epsilon$. For $\theta=1$, these complexities reduce to $\widetilde O(\epsilon^{-3})$ and $\widetilde O(\epsilon^{-4})$ respectively, which match, up to a logarithmic factor, the best-known complexities achieved by existing methods for finding an $\epsilon$-stochastic stationary point of unconstrained smooth stochastic optimization problems.

Finite sample deviation and variance bounds for first order autoregressive processes

A Finite-Sample Deviation Bound for Stable Autoregressive Processes

Limit Theory for Random Coefficient Autoregressive Process under Possibly Infinite Variance Error Sequence

Asymptotic Properties of the Least Squares Estimator in Local to Unity Processes with Fractional Gaussian Noise

Regularization and variable selection for infinite variance autoregressive models

The Variance of Partial Sums of Strong Near-Epoch Dependent Variables

Mildly Explosive Autoregression with Anti-persistent Errors

Sampling Requirements for Stable Autoregressive Estimation

Deviation Inequalities and Cramér-type Moderate Deviations for the Explosive Autoregressive Process

Precise Asymptotics for the First Moment of the Error Variance Estimator in Linear Models

Wasserstein distance bounds on the normal approximation of empirical autocovariances and cross-covariances under non-stationarity and stationarity

Variance-reduced first-order methods for deterministically constrained stochastic nonconvex optimization with strong convergence guarantees

On the Limiting Distribution of Sieve VAR($\infty$) Estimators in Small Samples

On the least squares estimator in a nearly unstable sequence of stationary spatial AR models

Almost Sure Central Limit Theorem for Error Variance Estimator in Pth-Order Nonlinear Autoregressive Processes

A weighted least squares procedure to approximate least absolute deviation estimation in time series with specific reference to infinite variance unit root problems

Statistical inference for the first-order autoregressive process with the fractional Gaussian noise

Strong confidence intervals for autoregression

Testing second order dynamics for autoregressive processes in presence of time-varying variance

Variance-Reduced Accelerated First-order Methods: Central Limit Theorems and Confidence Statements

Finite sample properties of estimators of spatial autoregressive models with autoregressive disturbances