Abstract:In this paper, we study unconstrained distributed optimization strongly convex problems, in which the exchange of information in the network is captured by a directed graph topology over digital channels that have limited capacity (and hence information should be quantized). Distributed methods in which nodes use quantized communication yield a solution at the proximity of the optimal solution, hence reaching an error floor that depends on the quantization level used; the finer the quantization the lower the error floor. However, it is not possible to determine in advance the optimal quantization level that ensures specific performance guarantees (such as achieving an error floor below a predefined threshold). Choosing a very small quantization level that would guarantee the desired performance, requires {information} packets of very large size, which is not desirable (could increase the probability of packet losses, increase delays, etc) and often not feasible due to the limited capacity of the channels available. In order to obtain a communication-efficient distributed solution and a sufficiently close proximity to the optimal solution, we propose a quantized distributed optimization algorithm that converges in a finite number of steps and is able to adjust the quantization level accordingly. The proposed solution uses a finite-time distributed optimization protocol to find a solution to the problem for a given quantization level in a finite number of steps and keeps refining the quantization level until the difference in the solution between two successive solutions with different quantization levels is below a certain pre-specified threshold.

Distributed Online Optimization Under Dynamic Adaptive Quantization

Distributed Constrained Optimization With Delayed Subgradient Information Over Time-Varying Network Under Adaptive Quantization

Distributed Optimization with Quantized Gradient Descent

Quantization Avoids Saddle Points in Distributed Optimization

Distributed Optimization via Gradient Descent with Event-Triggered Zooming over Quantized Communication

Distributed constrained optimization with periodic dynamic quantization

DQSGD: DYNAMIC QUANTIZED STOCHASTIC GRADIENT DESCENT FOR COMMUNICATION-EFFICIENT DISTRIBUTED LEARNING

Distributed Subgradient Method with Random Quantization and Flexible Weights: Convergence Analysis.

Distributed Optimization with Finite Bit Adaptive Quantization for Efficient Communication and Precision Enhancement

Error Compensated Quantized SGD and its Applications to Large-scale Distributed Optimization

Online Distributed Optimization with Nonconvex Objective Functions Via Dynamic Regrets

Augmented Distributed Gradient Methods for Multi-Agent Optimization under Uncoordinated Constant Stepsizes

DQ-SGD: Dynamic Quantization in SGD for Communication-Efficient Distributed Learning

Distributed Stochastic Algorithm for Global Optimization in Networked System

Convergence Theory of Generalized Distributed Subgradient Method with Random Quantization

Logarithmically Quantized Distributed Optimization over Dynamic Multi-Agent Networks

Double Quantization for Communication-Efficient Distributed Optimization

Online Distributed Optimization With Nonconvex Objective Functions Under Unbalanced Digraphs: Dynamic Regret Analysis

Online distributed optimization with stochastic gradients: high probability bound of regrets

Quantized Adam with Error Feedback

Distributed optimization for degenerate loss functions arising from over-parameterization