Abstract:This paper proposes a constrained stochastic successive convex approximation (CSSCA) algorithm to find a stationary point for a general non-convex stochastic optimization problem, whose objective and constraint functions are non-convex and involve expectations over random states. Most existing methods for non-convex stochastic optimization, such as the stochastic (average) gradient and stochastic majorization-minimization, only consider minimizing a stochastic non-convex objective over a deterministic convex set. The proposed CSSCA algorithm can also handle stochastic non-convex constraints in optimization problems, and it opens the way to solving more challenging optimization problems that occur in many applications. The algorithm is based on solving a sequence of convex objective/feasibility optimization problems obtained by replacing the objective/constraint functions in the original problems with some convex surrogate functions. The CSSCA algorithm allows a wide class of surrogate functions and thus provides many freedoms to design good surrogate functions for specific applications. Moreover, it also facilitates parallel implementation for solving large-scale stochastic optimization problems, which arise naturally in today's signal processing such as machine learning and big data analysis. We establish the convergence of the CSSCA algorithm with a feasible initial point, and customize the algorithmic framework to solve several important application problems. Simulations show that the CSSCA algorithm can achieve superior performance over existing solutions.

Sample Average Approximation for Stochastic Programming with Equality Constraints

Sample Average Approximation for Stochastic Programming with Equality Constraints

Nonasymptotic Upper Estimates for Errors of the Sample Average Approximation Method to Solve Risk-Averse Stochastic Programs

Sample Average Approximation Method for a Class of Stochastic Variational Inequality Problems.

On sample average approximation for two-stage stochastic programs without relatively complete recourse

Robust Stochastic Approximation Approach to Stochastic Programming

Sample Size Estimates for Risk-Neutral Semilinear PDE-Constrained Optimization

0/1 Constrained Optimization Solving Sample Average Approximation for Chance Constrained Programming

Metric Entropy-Free Sample Complexity Bounds for Sample Average Approximation in Convex Stochastic Programming

A 0/1 Constrained Optimization Solving Sample Average Approximation for Chance Constrained Programming

Bias Reduction in Sample-Based Optimization

Sample average approximation method for stochastic complementarity problems with applications to supply chain supernetworks

Adaptive and nonadaptive approaches to statistically based methods for solving stochastic linear programs: a computational investigation

A Gradually Reinforced Sample-Average-Approximation Differentiable Homotopy Method for a System of Stochastic Equations

Stochastic Nonlinear Complementarity Problems: Stochastic Programming Reformulation and Penalty-Based Approximation Method

Penalty-Based Saa Method of Stochastic Nonlinear Complementarity Problems

Multilevel Monte Carlo in Sample Average Approximation: Convergence, Complexity and Application

Stochastic Successive Convex Approximation for Non-Convex Constrained Stochastic Optimization

Risk-Averse Trajectory Optimization via Sample Average Approximation

Uniform exponential convergence of SAA with AMIS and asymptotics of its optimal value

Statistical inference and hypotheses testing of risk averse stochastic programs