Abstract:The goal of this paper is to study approaches to bridge the gap between first-order and second-order type methods for composite convex programs. Our key observations are: i) Many well-known operator splitting methods, such as forward-backward splitting (FBS) and Douglas-Rachford splitting (DRS), actually define a possibly semi-smooth and monotone fixed-point mapping; ii) The optimal solutions of the composite convex program and the solutions of the system of nonlinear equations derived from the fixed-point mapping are equivalent. Solving the system of nonlinear equations rediscovers a paradigm on developing second-order methods. Although these fixed-point mappings may not be differentiable, they are often semi-smooth and its generalized Jacobian matrix is positive semidefinite due to monotonicity. By combining a regularization approach and a known hyperplane projection technique, we propose an adaptive semi-smooth Newton method and establish its convergence to global optimality. A semi-smooth Levenberg-Marquardt (LM) method in terms of handling the nonlinear least squares formulation is further presented. In practice, the second-order methods can be activated until the first-order type methods reach a good neighborhood of the global optimal solution. Preliminary numerical results on Lasso regression, logistic regression, basis pursuit, linear programming and quadratic programming demonstrate that our second-order type algorithms are able to achieve quadratic or superlinear convergence as long as the fixed-point residual of the initial point is small enough.

A Globally Convergent Method Based on Fischer-Burmeister Operators for Solving Second-Order Cone Constrained Variational Inequality Problems

A Trust Region-Type Method for Solving Monotone Variational Inequality

A Least-Square Semismooth Newton Method for the Second-Order Cone Complementarity Problem

A Unified Convergence Analysis of a Second-Order Method of Multipliers for Nonlinear Conic Programming

A Mixed Line Search Smoothing Quasi-Newton Method for Solving Linear Second-Order Cone Programming Problem

A Linearly Convergent Derivative-Free Descent Method for the Second-Order Cone Complementarity Problem

A Smoothing Newton Method For Generalized Nash Equilibrium Problems With Second-Order Cone Constraints

A modified FR conjugate gradient method for solving linear second-order cone programming

Properties of Equation Reformulation of the Karush–Kuhn–Tucker Condition for Nonlinear Second Order Cone Optimization Problems

A Damped Gauss-Newton Method for the Second-Order Cone Complementarity Problem

A Homotopy Method for Nonlinear Second-Order Cone Programming

An Inexact Semismooth Newton Method for Variational Inequality with Symmetric Cone Constraints

Nonsingularity Conditions for the Fischer-Burmeister System of Nonlinear SDPs.

On the Coderivative of the Solution Mapping to a Second-Order Cone Constrained Parametric Variational Inequality

Global Convergence of an Inexact Operator Splitting Method for Monotone Variational Inequalities

A perturbation approach for an inverse linear second-order cone programming

A Krylov Subspace Method for Large-Scale Second-Order Cone Linear Complementarity Problem

Solving Nonconvex Optimal Control Problems by Convex Optimization

Semi-Smooth Second-order Type Methods for Composite Convex Programs

The Augmented Lagrangian Method for a Type of Inverse Quadratic Programming Problems over Second-Order Cones

ON THE INVERSE CONTINUOUS OPTIMIZATION AND ITS SMOOTHING FISCHER-BURMEISTER FUNCTION APPROACH