Abstract:Distributed optimization enables networked agents to cooperatively solve a global optimization problem even with each participating agent only having access to a local partial view of the objective function. Despite making significant inroads, most existing results on distributed optimization rely on noise-free information sharing among the agents, which is problematic when communication channels are noisy, messages are coarsely quantized, or shared information are obscured by additive noise for the purpose of achieving differential privacy. The problem of information-sharing noise is particularly pronounced in the state-of-the-art gradient-tracking based distributed optimization algorithms, in that information-sharing noise will accumulate with iterations on the gradient-tracking estimate of these algorithms, and the ensuing variance will even grow unbounded when the noise is persistent. This paper proposes a new gradient-tracking based distributed optimization approach that can avoid information-sharing noise from accumulating in the gradient estimation. The approach is applicable even when the {inter-agent interaction is} time-varying, which is key to enable the incorporation of a decaying factor in inter-agent interaction to gradually eliminate the influence of information-sharing noise. In fact, we rigorously prove that the proposed approach can ensure the almost sure convergence of all agents to the same optimal solution even in the presence of persistent information-sharing noise. The approach is applicable to general directed graphs. It is also capable of ensuring the almost sure convergence of all agents to an optimal solution when the gradients are noisy, which is common in machine learning applications. Numerical simulations confirm the effectiveness of the proposed approach.

Noise-robust Distributed Resource Allocation with Uncoordinated Step-Sizes

Exact Noise-robust Distributed Gradient-tracking Algorithm for Constraint-Coupled Resource Allocation Problems

A Distributed Algorithm With Event-Triggered Communication For Resource Allocation Problem

Distributed dual gradient tracking for economic dispatch in power systems with noisy information

Distributed Optimization For Resource Allocation With Event-Triggered Communication Under Directed Topology

Differentially Private Distributed Resource Allocation Via Deviation Tracking

An Exponentially Convergent Distributed Algorithm for Resource Allocation Problem

Gradient-tracking based Distributed Optimization with Guaranteed Optimality under Noisy Information Sharing

Disturbance-resilient Distributed Resource Allocation over Stochastic Networks using Uncoordinated Stepsizes

Distributed Dual Gradient Tracking for Resource Allocation in Unbalanced Networks

A Gradient Tracking Method for Resource Allocation Base on Distributed Convex Optimization

Augmented Distributed Gradient Methods for Multi-Agent Optimization under Uncoordinated Constant Stepsizes

A Dual Splitting Algorithm For Distributed Resource Allocation Problems

An Asynchronous Gradient Descent Based Method for Distributed Resource Allocation with Bounded Variables

Distributed Adaptive Gradient Algorithm with Gradient Tracking for Stochastic Non-Convex Optimization

ADMM-Tracking Gradient for Distributed Optimization over Asynchronous and Unreliable Networks

Ldecentralized Gradient Tracking For Continuous Dr-Submodular Maximization

Distributed Subgradient Algorithm for Multi-Agent Optimization with Dynamic Stepsize

Distributed Adaptive Event-Triggered Algorithms for Nonsmooth Resource Allocation Optimization Over Switching Topologies

A Dual Splitting Approach for Distributed Resource Allocation with Regularization