Abstract:Discrete optimization is a central problem in mathematical optimization with a broad range of applications, among which binary optimization and sparse optimization are two common ones. However, these problems are NP-hard and thus difficult to solve in general. Combinatorial search methods such as branch-and-bound and exhaustive search find the global optimal solution but are confined to small-sized problems, while coordinate descent methods such as coordinate gradient descent are efficient but often suffer from poor local minima. In this paper, we consider a hybrid method that combines the effectiveness of combinatorial search and the efficiency of coordinate descent. Specifically, we consider random strategy or/and greedy strategy to select a subset of coordinates as the working set, and then perform global combinatorial search over the working set based on the original objective function. In addition, we provide some optimality analysis and convergence analysis for the proposed method. Our method finds stronger stationary points than existing methods. Finally, we demonstrate the efficacy of our method on some sparse optimization and binary optimization applications. As a result, our method achieves state-of-the-art performance in terms of accuracy. For example, our method generally outperforms the well-known orthogonal matching pursuit method in sparse optimization. Keywords: Sparsity Optimization, Binary Optimization, Coordinate Descent, Combinatorial Search, Discrete Optimization, Convergence Analysis.

A Hybrid Algorithm for Convex Semidefinite Optimization

A conditional gradient homotopy method with applications to Semidefinite Programming

A New Hybrid Descent Algorithm for Large-Scale Nonconvex Optimization and Application to Some Image Restoration Problems

A Conditional Gradient Framework for Composite Convex Minimization with Applications to Semidefinite Programming

Low-Rank Extragradient Methods for Scalable Semidefinite Optimization

Efficient Convex Algorithms for Universal Kernel Learning

Distributed Synchronous and Asynchronous Algorithms for Semidefinite Programming With Diagonal Constraints

Almost Optimal Sublinear Time Algorithm for Semidefinite Programming

A Hybrid Method of Combinatorial Search and Coordinate Descent for Discrete Optimization

Post-Processing with Projection and Rescaling Algorithms for Semidefinite Programming

Computation Algorithm for Convex Semi-infinite Program with Second-Order Cones: Special Analyses for Affine and Quadratic Case

A predictor-corrector algorithm for semidefinite programming that uses the factor width cone

A Primal-Dual Framework for Symmetric Cone Programming

Efficient Low-Rank Semidefinite Programming With Robust Loss Functions

A Krylov Subspace Method for Large-Scale Second-Order Cone Linear Complementarity Problem

An accelerated active-set algorithm for a quadratic semidefinite program with general constraints

Subgradient-Based Neural Network for Nonconvex Optimization Problems in Support Vector Machines with Indefinite Kernels

A semi-conjugate gradient method for solving unsymmetric positive definite linear systems

A New Hybrid Algorithm for Solving Large Scale Global Optimization Problems

A Nonlinear Bregman Primal-Dual Framework for Optimizing Nonconvex Infimal Convolutions

Homotopy Methods for Convex Optimization