Abstract:Conic programming has well-documented merits in a gamut of signal processing and machine learning tasks. This contribution revisits a recently developed first-order conic descent (CD) solver, and advances it in three aspects: intuition, theory, and algorithmic implementation. It is found that CD can afford an intuitive geometric derivation that originates from the dual problem. This opens the door to novel algorithmic designs, with a momentum variant of CD, momentum conic descent (MOCO) exemplified. Diving deeper into the dual behavior CD and MOCO reveals: i) an analytically justified stopping criterion; and, ii) the potential to design preconditioners to speed up dual convergence. Lastly, to scale semidefinite programming (SDP) especially for low-rank solutions, a memory efficient MOCO variant is developed and numerically validated.

What problem does this paper attempt to address?

The main problems that this paper attempts to solve are the memory efficiency and convergence speed issues faced by optimization methods when dealing with large - scale semidefinite programming (SDP) problems. Specifically: 1. **Memory Efficiency**: For large - scale SDP problems, traditional optimization methods need to store and operate large matrices, which leads to huge memory consumption. The paper proposes a memory - efficient MOCO (Momentum Conic Descent) algorithm, which reduces memory usage by using vectorized representations and random sketching techniques. 2. **Convergence Speed**: Traditional optimization methods usually only focus on the convergence of the original problem and ignore the properties of the dual problem. By in - depth study of dual convergence, the paper proposes the MOCO algorithm with momentum to accelerate the convergence speed and provides a stopping criterion based on dual feasibility. ### Main Contributions 1. **Geometric Interpretation**: - The paper provides a geometric interpretation of the Cone Descent (CD) method, revealing its mathematical basis in the dual domain. - This interpretation is not only intuitive but also provides new opportunities for algorithm design, thus leading to the proposal of the MOCO algorithm with momentum. 2. **Algorithm Improvement**: - The MOCO algorithm is proposed, which accelerates the convergence speed by introducing momentum. - The dual convergence of MOCO is analyzed in detail, providing theoretical support. 3. **Stopping Criterion**: - Based on dual feasibility, a practical stopping criterion is proposed to estimate sub - optimality. 4. **Memory - Efficient Implementation**: - For large - scale SDP problems, a memory - efficient MOCO implementation method is proposed. - By using vectorized representations and random sketching techniques, the memory consumption is significantly reduced. - Combined with the Burer - Monteiro (BM) heuristic method, the numerical performance is further improved. ### Theoretical Analysis - **Original Convergence**: It is proved that the MOCO algorithm has good original convergence under certain assumptions. - **Dual Convergence**: The dual convergence of MOCO is analyzed in detail, revealing that the dual behavior can be influenced by pre - conditioning. - **Stopping Criterion**: A stopping criterion based on dual feasibility is provided to ensure that the algorithm can be terminated when a certain precision is reached. ### Experimental Verification - The effectiveness of the MOCO algorithm is verified through numerical experiments. In particular, when dealing with SDP problems with low - rank solutions, the memory - efficient MOCO performs especially well. Overall, through theoretical analysis and experimental verification, the paper proposes an optimization method that is memory - efficient and has a fast convergence speed, which is suitable for large - scale semidefinite programming problems.

Conic Descent Redux for Memory-Efficient Optimization

A Cyclic Coordinate Descent Method for Convex Optimization on Polytopes

Efficiency of Coordinate Descent Methods For Structured Nonconvex Optimization

A conditional gradient homotopy method with applications to Semidefinite Programming

Limited-memory common-directions method for large-scale optimization: convergence, parallelization, and distributed optimization

A Primal-Dual Framework for Symmetric Cone Programming

Convergence Depth Control for Process System Optimization

Convergence depth control for interior point methods

Second-Order Conic and Polyhedral Approximations of the Exponential Cone: Application to Mixed-Integer Exponential Conic Programs

Hamiltonian Descent and Coordinate Hamiltonian Descent

Leveraging Memory Effects and Gradient Information in Consensus-Based Optimization: On Global Convergence in Mean-Field Law

Conic Approximation to Quadratic Optimization with Linear Complementarity Constraints.

Accelerated Cyclic Coordinate Dual Averaging with Extrapolation for Composite Convex Optimization

Conic approximation to nonconvex quadratic programming with convex quadratic constraints

Dual Lagrangian Learning for Conic Optimization

Dual Conic Proxies for AC Optimal Power Flow

Further Development in Convex Conic Reformulation of Geometric Nonconvex Conic Optimization Problems

Exploiting cone approximations in an augmented Lagrangian method for conic optimization

Simplex QP-based methods for minimizing a conic quadratic objective over polyhedra

Efficient Accelerated Coordinate Descent Methods and Faster Algorithms for Solving Linear Systems

Mixed-Projection Conic Optimization: A New Paradigm for Modeling Rank Constraints