Abstract:This paper proposes a constrained stochastic successive convex approximation (CSSCA) algorithm to find a stationary point for a general non-convex stochastic optimization problem, whose objective and constraint functions are non-convex and involve expectations over random states. Most existing methods for non-convex stochastic optimization, such as the stochastic (average) gradient and stochastic majorization-minimization, only consider minimizing a stochastic non-convex objective over a deterministic convex set. The proposed CSSCA algorithm can also handle stochastic non-convex constraints in optimization problems, and it opens the way to solving more challenging optimization problems that occur in many applications. The algorithm is based on solving a sequence of convex objective/feasibility optimization problems obtained by replacing the objective/constraint functions in the original problems with some convex surrogate functions. The CSSCA algorithm allows a wide class of surrogate functions and thus provides many freedoms to design good surrogate functions for specific applications. Moreover, it also facilitates parallel implementation for solving large-scale stochastic optimization problems, which arise naturally in today's signal processing such as machine learning and big data analysis. We establish the convergence of the CSSCA algorithm with a feasible initial point, and customize the algorithmic framework to solve several important application problems. Simulations show that the CSSCA algorithm can achieve superior performance over existing solutions.

Sequential Convex Approximations to Joint Chance Constrained Programs: A Monte Carlo Approach.

A Smooth Monte Carlo Approach to Joint Chance-Constrained Programs

Gradient and Hessian of Joint Probability Function with Applications on Chance-Constrained Programs

Approximating Data-Driven Joint Chance-Constrained Programs Via Uncertainty Set Construction

Nonconvex and Nonsmooth Approaches for Affine Chance-Constrained Stochastic Programs

Solving a Class of Convex Stochastic Programs Via Monte Carlo Simulation

Bayesian Joint Chance Constrained Optimization: Approximations and Statistical Consistency

Stochastic Successive Convex Approximation for Non-Convex Constrained Stochastic Optimization

Solving Nonsmooth Nonconvex Compound Stochastic Programs with Applications to Risk Measure Minimization

Conditional Value-at-Risk Approximation to Value-at-Risk Constrained Programs: A Remedy Via Monte Carlo.

Joint Chance Constrained Optimal Control via Linear Programming

Multiple Joint Chance Constraints Approximation for Uncertainty Modeling in Dispatch Problems

Conformal Predictive Programming for Chance Constrained Optimization

Constraint Qualification, the Strong CHIP, and Best Approximation with Convex Constraints in Banach Spaces.

Robust approximation of chance constrained optimization with polynomial perturbation

Optimizing Chance-Constrained Submodular Problems with Variable Uncertainties

A D.C. approximation approach for optimization with probabilistic constraints based on Chen–Harker–Kanzow–Smale smooth plus function

Chance Constrained Probability Measure Optimization: Problem Formulation, Equivalent Reduction, and Sample-based Approximation

Stochastic Approximation for Expectation Objective and Expectation Inequality-Constrained Nonconvex Optimization

An Empirical Quantile Estimation Approach to Nonlinear Optimization Problems with Chance Constraints