Abstract:Coordinate update/descent algorithms are widely used in large-scale optimization due to their low per-iteration cost and scalability, but their behavior on infeasible or misspecified problems has not been much studied compared to the algorithms that use full updates. For coordinate-update methods to be as widely adopted to the extent so that they can be used as engines of general-purpose solvers, it is necessary to also understand their behavior under pathological problem instances. In this work, we show that the normalized iterates of randomized coordinate-update fixed-point iterations (RC-FPI) converge to the infimal displacement vector and use this result to design an efficient infeasibility detection method. We then extend the analysis to the setup where the coordinates are defined by non-orthonormal basis using the Friedrichs angle and then apply the machinery to decentralized optimization problems.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is the behavior analysis of the coordinate update algorithm in infeasible or mis - specified optimization problems. Specifically, the author focuses on the performance of the Random Coordinate - Update Fixed - Point Iteration (RC - FPI) when dealing with inconsistent problem instances and proposes an effective infeasibility detection method. ### Background and Motivation - **Background**: Coordinate update/descent algorithms are widely used in large - scale optimization problems because of their low per - iteration cost and scalability. However, the behavior of these algorithms when dealing with infeasible or mis - specified problems has not been fully studied. - **Motivation**: In order to make the coordinate update method as widely applicable to general - purpose optimization solvers as the full - update method, it is necessary to understand their behavior under ill - conditioned problem instances. ### Main Contributions 1. **Theoretical Analysis**: - It is proved that the normalized iteration \( \frac{x_k}{k} \) converges in \( L^2 \) and almost surely to the minimum displacement vector \( -\alpha v \), where \( v \) is the minimum displacement vector of the inconsistent operator \( \mathbb{T} \). - The asymptotic bias and variance of the normalized iteration are analyzed, and the asymptotic upper bounds of the bias and variance are provided. 2. **Infeasibility Detection Method**: - Using the above theoretical results, an efficient infeasibility detection method is designed, which judges whether the problem is infeasible by checking \( \frac{1}{k}\|x_k\| > \epsilon \). 3. **Extended Applications**: - The analysis is extended to the coordinate system defined by non - orthogonal bases, and the Friedrichs angle is used for analysis. - It is applied to decentralized optimization problems. ### Key Concepts - **Minimum Displacement Vector**: For an inconsistent operator \( \mathbb{T} \), its minimum displacement vector \( v \) is defined as \( v=\arg\min_{y\in\text{range}(\mathbb{I}-\mathbb{T})}\|y\|_M \). - **Normalized Iteration**: Defined as \( \frac{x_k}{k} \), where \( x_k \) is the result of the \( k \)-th iteration. - **Random Coordinate - Update Fixed - Point Iteration (RC - FPI)**: Only a randomly selected set of coordinates is updated at each iteration. ### Conclusion Through theoretical analysis and experimental verification, this paper shows the effectiveness and robustness of the Random Coordinate - Update Fixed - Point Iteration in dealing with infeasible problems and provides a practical infeasibility detection method. This provides theoretical support and practical tools for applying the coordinate update method to a wider range of optimization problems.

Coordinate-Update Algorithms can Efficiently Detect Infeasible Optimization Problems

Coordinate-friendly Structures, Algorithms and Applications

Cyclic Coordinate-Update Algorithms for Fixed-Point Problems: Analysis and Applications.

Inexact Coordinate Descent: Complexity and Preconditioning

On convergence of a $q$-random coordinate constrained algorithm for non-convex problems

A framework of constraint preserving update schemes for optimization on Stiefel manifold

A Hybrid Method of Combinatorial Search and Coordinate Descent for Discrete Optimization

Online Convex Optimization Using Coordinate Descent Algorithms

Task Embedded Coordinate Update: A Realizable Framework for Multivariate Non-Convex Optimization.

A Globally Convergent Algorithm for Nonconvex Optimization Based on Block Coordinate Update

Efficient Accelerated Coordinate Descent Methods and Faster Algorithms for Solving Linear Systems

Fast stochastic dual coordinate descent algorithms for linearly constrained convex optimization

Efficiency of Coordinate Descent Methods For Structured Nonconvex Optimization

A Cyclic Coordinate Descent Method for Convex Optimization on Polytopes

A Block Coordinate Descent Method for Nonsmooth Composite Optimization under Orthogonality Constraints

Adaptive block coordinate DIRECT algorithm

A Flexible Coordinate Descent Method

An Optimization Approach for Team Coordination Through Information Sharing

Pathwise coordinate optimization

Structured Nonconvex and Nonsmooth Optimization: Algorithms and Iteration Complexity Analysis