Abstract:Tremendous efficient optimization methods have been proposed for strongly convex objectives optimization in modern machine learning. For non-strongly convex objectives, a popular approach is to apply a reduction from non-strongly convex to a strongly convex case via regularization techniques. Reduction on objectives with adaptive decrease on regularization tightens the optimal convergence of algorithms to be independent on logarithm factor. However, the initialization of parameter of regularization has a great impact on the performance of the reduction. In this paper, we propose an aggressive reduction to reduce the complexity of optimization for non-strongly convex objectives, and our reduction eliminates the impact of the initialization of parameter on the convergent performances of algorithms. Aggressive reduction not only adaptively decreases the regularization parameter, but also modifies regularization term as the distance between current point and the approximate minimizer. Our aggressive reduction can also shave off the non-optimal logarithm term theoretically, and make the convergent performance of algorithm more compact practically. Experimental results on logistic regression and image deblurring confirm this success in practice.

A new bound-and-reduce approach of nonconvex quadratic programming problems

A GLOBALLY AND SUPERLIEARLY CONVERGENT FEASIBLE SQP ALGORITHM FOR DEGENERATE CONSTRAINED OPTIMIZATION

Solving Convex Quadratic Programming by Potential-Reduction Interior-Point Algorithm

An Efficient Interior-Point Algorithm with New Non-Monotone Line Search Filter Method for Nonlinear Constrained Programming

A new variable reduction technique for convex integer quadratic programs

A New Global Optimization Method Based on Simplex Branching for Solving a Class of Non-Convex QCQP Problems

An Eigenvalue Decomposition Based Branch-and-bound Algorithm for Nonconvex Quadratic Programming Problems with Convex Quadratic Constraints

A New Algorithm for Finding Global Solution of Indefinite Quadratic Programming Problems

An Aggressive Reduction on the Complexity of Optimization for Non-Strongly Convex Objectives

An Improved Sequential Quadratic Programming Algorithm for Solving General Nonlinear Programming Problems

Separable Relaxation for Nonconvex Quadratic Integer Programming: Integer Diagonalization Approach

Quadratic convex reformulation for quadratic programming with linear on-off constraints.

Global solutions of nonconvex standard quadratic programs via mixed integer linear programming reformulations

An efficient branch and bound reduction algorithm for globally solving linear fractional programming problems

An accelerated active-set algorithm for a quadratic semidefinite program with general constraints

A Sequential Quadratic Programming Method with High Probability Complexity Bounds for Nonlinear Equality Constrained Stochastic Optimization

Conic approximation to nonconvex quadratic programming with convex quadratic constraints

Convex Relaxation and Lagrangian Decomposition for Indefinite Integer Quadratic Programming

On the Reduction of Duality Gap in Box Constrained Nonconvex Quadratic Program

Convex Relaxations for Nonconvex Quadratically Constrained Quadratic Programming: Matrix Cone Decomposition and Polyhedral Approximation

Compact mixed-integer programming formulations in quadratic optimization