Abstract:Distributed learning and adaptation have received significant interest and found wide-ranging applications in machine learning and signal processing. While various approaches, such as shared-memory optimization, multi-task learning, and consensus-based learning (e.g., federated learning and learning over graphs), focus on optimizing either local rewards or a global reward, there remains a need for further exploration of their interconnections. This paper specifically focuses on a scenario where agents collaborate towards a common task (i.e., optimizing a global reward equal to aggregated local rewards) while effectively having distinct individual tasks (i.e., optimizing individual local parameters in a local reward). Each agent's actions can potentially impact other agents' performance through interactions. Notably, each agent has access to only its local zeroth-order oracle (i.e., reward function value) and shares scalar values, rather than gradient vectors, with other agents, leading to communication bandwidth efficiency and agent privacy. Agents employ zeroth-order optimization to update their parameters, and the asynchronous message-passing between them is subject to bounded but possibly random communication delays. This paper presents theoretical convergence analyses and establishes a convergence rate for nonconvex problems. Furthermore, it addresses the relevant use-case of deep learning-based resource allocation in communication networks and conducts numerical experiments in which agents, acting as transmitters, collaboratively train their individual policies to maximize a global reward, e.g., a sum of data rates.

Distributed Zeroth-Order Optimization: Convergence Rates That Match Centralized Counterpart

Zeroth-order algorithms for stochastic distributed nonconvex optimization

Single Point-Based Distributed Zeroth-Order Optimization with a Non-Convex Stochastic Objective Function

Hybrid Decentralized Optimization: Leveraging Both First- and Zeroth-Order Optimizers for Faster Convergence

A Zeroth-Order Variance-Reduced Method for Decentralized Stochastic Non-convex Optimization

Zeroth-Order Non-Convex Optimization for Cooperative Multi-Agent Systems with Diminishing Step Size and Smoothing Radius

Finite-time Distributed ConvexOptimization with Zero-Gradient-Sum Algorithms

Linear Convergence of First- and Zeroth-Order Primal-Dual Algorithms for Distributed Nonconvex Optimization

Zeroth-Order Feedback Optimization in Multi-Agent Systems: Tackling Coupled Constraints

Communication-Efficient Decentralized Zeroth-order Method on Heterogeneous Data

Distributed Optimization Algorithm with Superlinear Convergence Rate

Designing Zero-Gradient-Sum Protocols for Finite-Time Distributed Optimization Problem

Asynchronous Message-Passing and Zeroth-Order Optimization Based Distributed Learning with a Use-Case in Resource Allocation in Communication Networks

Distributed Algorithms for Composite Optimization: Unified Framework and Convergence Analysis

Accelerated Primal-Dual Algorithms for Distributed Smooth Convex Optimization over Networks

Zeroth-Order Feedback-Based Optimization for Distributed Demand Response

Online Optimization Perspective on First-Order and Zero-Order Decentralized Nonsmooth Nonconvex Stochastic Optimization

Distributed Interval Optimization with Stochastic Zeroth-order Oracle

A Zeroth-Order Proximal Algorithm for Consensus Optimization

Convergence in High Probability of Distributed Stochastic Gradient Descent Algorithms

Linearly Convergent Second-Order Distributed Optimization Algorithms