Abstract:The next-generation of wireless networks will enable many machine learning (ML) tools and applications to efficiently analyze various types of data collected by edge devices for inference, autonomy, and decision making purposes. However, due to resource constraints, delay limitations, and privacy challenges, edge devices cannot offload their entire collected datasets to a cloud server for centrally training their ML models or inference purposes. To overcome these challenges, distributed learning and inference techniques have been proposed as a means to enable edge devices to collaboratively train ML models without raw data exchanges, thus reducing the communication overhead and latency as well as improving data privacy. However, deploying distributed learning over wireless networks faces several challenges including the uncertain wireless environment (e.g., dynamic channel and interference), limited wireless resources (e.g., transmit power and radio spectrum), and hardware resources (e.g., computational power). This paper provides a comprehensive study of how distributed learning can be efficiently and effectively deployed over wireless edge networks. We present a detailed overview of several emerging distributed learning paradigms, including federated learning, federated distillation, distributed inference, and multi-agent reinforcement learning. For each learning framework, we first introduce the motivation for deploying it over wireless networks. Then, we present a detailed literature review on the use of communication techniques for its efficient deployment. We then introduce an illustrative example to show how to optimize wireless networks to improve its performance. Finally, we introduce future research opportunities. In a nutshell, this paper provides a holistic set of guidelines on how to deploy a broad range of distributed learning frameworks over real-world wireless communication networks.

Guest Editorial Introduction to the Special Section on Communication-Efficient Distributed Machine Learning

Efficient Partitioning and Communication Scheme-Based Distributed Edge Computing to Accelerate Deep Neural Network

Guest Editorial: Communication Technologies for Efficient Edge Learning

Communication-Efficient Distributed Deep Learning: A Comprehensive Survey

Communication-Efficient Large-Scale Distributed Deep Learning: A Comprehensive Survey

An Introduction to Communication Efficient Edge Machine Learning

DISTRIBUTED HIGH-PERFORMANCE COMPUTING METHODS FOR ACCELERATING DEEP LEARNING TRAINING

Communication Patterns in Distributed Deep Learning

A Survey From Distributed Machine Learning to Distributed Deep Learning

Communication Optimization for Distributed Training: Architecture, Advances, and Opportunities

A Systematic Literature Review on Distributed Machine Learning in Edge Computing

IEEE TCCN Special Section Editorial: Convergence of Collaborative Distributed Machine Learning and Edge Computing Toward Intelligent Networking

A Quantitative Survey of Communication Optimizations in Distributed Deep Learning

Guest Editorial: Sustainable Big AI Model for Wireless Networks

Distributed Machine Learning for Wireless Communication Networks: Techniques, Architectures, and Applications

Distributed Learning Systems with First-order Methods

Distributed Learning in Wireless Networks: Recent Progress and Future Challenges

Communication Efficient Distributed Training with Distributed Lion

Distributed Intelligence in Wireless Networks

A Survey of Distributed Learning in Cloud, Mobile, and Edge Settings

Peering Beyond the Gradient Veil with Distributed Auto Differentiation