Abstract:Distributed optimization aims to effectively complete specified tasks through cooperation among multi- agent systems, which has achieved great success in large-scale optimization problems. However, it remains a challenging task to develop an effective distributed algorithm with theoretical guarantees, especially when dealing with nonconvex constraints. More importantly, high-dimensional data often exhibits inherent structures such as sparsity, which if exploited accurately, can significantly enhance the capture of its intrinsic characteristics. In this paper, we introduce a novel distributed sparsity constrained optimization framework over the Stiefel manifold, abbreviated as DREAM. DREAM innovatively integrates the t 2 , 0-norm constraint and Stiefel manifold constraint within a distributed optimization setting, which has not been investigated in existing literature. Unlike the existing distributed methods, the proposed DREAM not only can extract the similarity information among samples, but also more flexibly determine the number of features to be extracted. Then, we develop an efficient Newton augmented Lagrangian-based algorithm. In theory, we delve into the relationship between the minimizer, the Karush-Kuhn-Tucker point, and the stationary point, and rigorously demonstrate that the sequence generated by our algorithm converges to a stationary point. Extensive numerical experiments verify its superiority over state-of-the-art distributed methods.

Distributed Optimization and Sampling with Constraints from Machine Learning

Distributed Sparsity Constrained Optimization over the Stiefel Manifold

The Geometric Effects of Distributing Constrained Nonconvex Optimization Problems

Global Optimization with Orthogonality Constraints Via Stochastic Diffusion on Manifold

Optimization on manifolds: A symplectic approach

Fast Convergence of Langevin Dynamics on Manifold: Geodesics meet Log-Sobolev

Optimization on Multiple Manifolds

Data driven Dirichlet sampling on manifolds

Extrinsic Bayesian Optimizations on Manifolds

Communication Efficient Parallel Algorithms for Optimization on Manifolds

Topology Optimization on Two-Dimensional Manifolds

Categorical Structural Optimization Using Discrete Manifold Learning Approach and Custom-Built Evolutionary Operators

From the Simplex to the Sphere: Faster Constrained Optimization Using the Hadamard Parametrization

Distributed Learning with Convex SUM-of -Non-convex Objective

Learning the Solution Manifold in Optimization and Its Application in Motion Planning

Distributed Learning via Filtered Hyperinterpolation on Manifolds

Optimizations on Unknown Low-Dimensional Structures Given by High-Dimensional Data

Nonsmooth Optimization over Stiefel Manifold: Riemannian Subgradient Methods

From Optimization to Sampling via Lyapunov Potentials

Stochastic Gradient Geodesic MCMC Methods.

Distributed Optimization with Projection-free Dynamics