Abstract:Decentralized federated learning (DFL) is a variant of federated learning, where edge nodes only communicate with their one-hop neighbors to learn the optimal model. However, as information exchange is restricted in a range of one-hop in DFL, inefficient information exchange leads to more communication rounds to reach the targeted training loss. This greatly reduces the communication efficiency. In this paper, we propose a new non-uniform quantization of model parameters to improve DFL convergence. Specifically, we apply the Lloyd-Max algorithm to DFL (LM-DFL) first to minimize the quantization distortion by adjusting the quantization levels adaptively. Convergence guarantee of LM-DFL is established without convex loss assumption. Based on LM-DFL, we then propose a new doubly-adaptive DFL, which jointly considers the ascending number of quantization levels to reduce the amount of communicated information in the training and adapts the quantization levels for non-uniform gradient distributions. Experiment results based on MNIST and CIFAR-10 datasets illustrate the superiority of LM-DFL with the optimal quantized distortion and show that doubly-adaptive DFL can greatly improve communication efficiency.

What problem does this paper attempt to address?

### Problems Addressed by the Paper This paper aims to address the communication efficiency issue in Decentralized Federated Learning (DFL). Specifically, in DFL, information exchange between nodes is limited to one-hop neighbors, leading to inefficient information exchange and requiring more communication rounds to achieve the target training loss. This significantly reduces communication efficiency. To improve the convergence speed and communication efficiency of DFL, the authors propose a new non-uniform quantization method to optimize model parameters. Specifically, they first apply the Lloyd-Max algorithm (LM-DFL) to minimize quantization distortion and adaptively reduce communication volume by adjusting the quantization levels. On this basis, the authors further propose a dual adaptive DFL framework that jointly considers the number of quantization levels and non-uniform gradient distribution to further improve communication efficiency. ### Main Contributions 1. **Development of LM Vector Quantizer**: Designed an LM vector quantizer suitable for DFL, which minimizes quantization distortion given the number of quantization levels. 2. **Convergence Analysis of Quantization Distortion**: Established a general convergence bound for DFL with arbitrary quantization or compression operators without assuming convex loss functions. Extended the results to the proposed LM-DFL, providing its convergence performance. 3. **Proposed Dual Adaptive DFL Algorithm**: Based on the convergence bound of LM-DFL, proposed a dual adaptive algorithm that adaptively adjusts the quantization levels and the number of quantization levels to optimize convergence performance by matching the time-varying convergence rate and variable gradient distribution. ### Experimental Results Experimental results show that LM-DFL performs excellently on the MNIST and CIFAR-10 datasets, with optimal quantization distortion. The dual adaptive DFL can significantly improve communication efficiency. ### Conclusion By introducing the LM vector quantizer and the dual adaptive DFL framework, this paper effectively addresses the communication efficiency issue in DFL, improving the model's convergence speed and communication efficiency. These methods have significant value in practical applications.

Communication-Efficient Design for Quantized Decentralized Federated Learning

Communication-Efficient Design for Quantized Decentralized Federated Learning

FedDQ: Communication-Efficient Federated Learning with Descending Quantization

Lazily Aggregated Quantized Gradient Innovation for Communication-Efficient Federated Learning.

FedAQ: Communication-Efficient Federated Edge Learning via Joint Uplink and Downlink Adaptive Quantization

Decentralized Federated Learning: Balancing Communication and Computing Costs

General Decentralized Federated Learning for Communication-Computation Tradeoff.

Design and Analysis of Uplink and Downlink Communications for Federated Learning

Communication Efficient Federated Learning with Adaptive Quantization

Rate distortion optimization for adaptive gradient quantization in federated learning

NQFL: Nonuniform Quantization for Communication Efficient Federated Learning

Quantization Bits Allocation for Wireless Federated Learning

Two-layer accumulated quantized compression for communication-efficient federated learning: TLAQC

A Communication-Efficient Federated Learning by Dynamic Quantization and Free-Ride Coding

FedFQ: Federated Learning with Fine-Grained Quantization

Ternary Compression for Communication-Efficient Federated Learning.

Communication-Efficient Federated Learning via Clipped Uniform Quantization

Hierarchical Quantized Federated Learning: Convergence Analysis and System Design

Hierarchical Federated Learning with Quantization: Convergence Analysis and System Design

Secure and Efficient Federated Learning with Provable Performance Guarantees Via Stochastic Quantization

Decentralized Federated Learning Over Imperfect Communication Channels