Abstract: In this paper, we consider the problem of stochastic optimization, where the objective function is in terms of the expectation of a (possibly non-convex) cost function that is parametrized by a random variable. While the convergence speed is critical for many emerging applications, most existing stochastic optimization methods suffer from slow convergence. Furthermore, the emerging technology of parallel computing has motivated an increasing demand for designing new stochastic optimization schemes that can handle parallel optimization for implementation in distributed systems. We propose a fast parallel stochastic optimization framework that can solve a large class of possibly non-convex stochastic optimization problems that may arise in applications with multi-agent systems. In the proposed method, each agent updates its control variable in parallel, by solving a convex quadratic subproblem independently. The convergence of the proposed method to the optimal solution for convex problems and to a stationary point for general non-convex problems is established. The proposed algorithm can be applied to solve a large class of optimization problems arising in important applications from various fields, such as machine learning and wireless networks. As a representative application of our proposed stochastic optimization framework, we focus on large-scale support vector machines and demonstrate how our algorithm can efficiently solve this problem, especially in modern applications with huge datasets. Using popular real-world datasets, we present experimental results to demonstrate the merits of our proposed framework by comparing its performance to the state-of-the-art in the literature. Numerical results show that the proposed method can significantly outperform the state-of-the-art methods in terms of the convergence speed while having the same or lower complexity and storage requirement.

A Fast Non-Monte-Carlo Yield Analysis and Optimization by Stochastic Orthogonal Polynomials

Parallel Stochastic Optimization Framework for Large-Scale Non-Convex Stochastic Problems

Design Time Body Bias Selection for Parametric Yield Improvement.

Yield Optimization based on Adaptive Newton-Monte Carlo and Polynomial Surrogates

An Efficient SRAM Yield Analysis and Optimization Method with Adaptive Online Surrogate Modeling

An Efficient and Robust Yield Optimization Method for High-dimensional SRAM Circuits.

An Efficient Yield Optimization Method for Analog Circuits Via Gaussian Process Classification and Varying-Sigma Sampling

A Hybrid Dynamic Optimization Approach for Stability Constrained Optimal Power Flow

Efficient Yield Optimization for Analog and SRAM Circuits Via Gaussian Process Regression and Adaptive Yield Estimation

Quickyield: An Efficient Global-Search Based Parametric Yield Estimation With Performance Constraints

Performance Bound and Yield Analysis for Analog Circuits under Process Variations.

Yield Maximization of Flip-Flop Circuits Based on Deep Neural Network and Polyhedral Estimation of Nonlinear Constraints

Yield Optimization using Hybrid Gaussian Process Regression and a Genetic Multi-Objective Approach

Polynomial Chaos-Based Approach to Yield-Driven EM Optimization

A Yield-Enhanced Global Optimization Methodology for Analog Circuit Based on Extreme Value Theory

A Fast Methodology for Topology Optimizations of Electromagnetic Devices

Analog Circuit Yield Optimization Via Freeze–Thaw Bayesian Optimization Technique

Reliability-Based and Cost-Oriented Product Optimization Integrating Fuzzy Reasoning Petri Nets, Interval Expert Evaluation and Cultural-Based DMOPSO Using Crowding Distance Sorting

Bayesian Methods for the Yield Optimization of Analog and SRAM Circuits

Efficient Bayesian Yield Optimization Approach for Analog and SRAM Circuits

C-Yes: an Efficient Parametric Yield Estimation Approach for Analog and Mixed-Signal Circuits Based on Multicorner-Multiperformance Correlations