Abstract:Efficient collaboration between collaborative machine learning and wireless communication technology, forming a Federated Edge Learning (FEEL), has spawned a series of next-generation intelligent applications. However, due to the openness of network connections, the FEEL framework generally involves hundreds of remote devices (or clients), resulting in expensive communication costs, which is not friendly to resource-constrained FEEL. To address this issue, we propose a distributed approximate Newton-type algorithm with fast convergence speed to alleviate the problem of FEEL resource (in terms of communication resources) constraints. Specifically, the proposed algorithm is improved based on distributed L-BFGS algorithm and allows each client to approximate the high-cost Hessian matrix by computing the low-cost Fisher matrix in a distributed manner to find a better descent direction, thereby speeding up convergence. Second, we prove that the proposed algorithm has linear convergence in strongly convex and non-convex cases and analyze its computational and communication complexity. Similarly, due to the heterogeneity of the connected remote devices, FEEL faces the challenge of heterogeneous data and non-IID (Independent and Identically Distributed) data. To this end, we design a simple but elegant training scheme, namely FedOVA (Federated One-vs-All), to solve the heterogeneous statistical challenge brought by heterogeneous data. In this way, FedOVA first decomposes a multi-class classification problem into more straightforward binary classification problems and then combines their respective outputs using ensemble learning. In particular, the scheme can be well integrated with our communication efficient algorithm to serve FEEL. Numerical results verify the effectiveness and superiority of the proposed algorithm.

Fast Personalized Federated Learning in Wireless Networks With Heterogeneous Data and Limited Communication Resources

Optimize Scheduling of Federated Learning on Battery-powered Mobile Devices

Fast Convergence for Federated Learning in OFDMA Systems

Analysis and Optimization of Wireless Federated Learning with Data Heterogeneity

Adaptive Batchsize Selection and Gradient Compression for Wireless Federated Learning

Adaptive Model Pruning and Personalization for Federated Learning Over Wireless Networks

Asynchronous Federated Learning over Wireless Communication Networks.

Hierarchical Federated Learning with Adaptive Clustering on Non-IID Data

Over-the-Air Hierarchical Personalized Federated Learning

Joint User Association and Resource Allocation for Wireless Hierarchical Federated Learning with IID and Non-IID Data

Joint User Selection and Resource Allocation for Fast Federated Edge Learning

Adaptive User Selection and Bandwidth Allocation for Fast Convergence of Federated Learning in Wireless Networks

Federated Learning over Wireless Networks: Optimization Model Design and Analysis

Personalized Federated Learning towards Communication Efficiency, Robustness and Fairness

Adaptive Hierarchical Federated Learning Over Wireless Networks

Resource-constrained Federated Learning with Heterogeneous Data: Formulation and Analysis

Min-Max Cost Optimization for Efficient Hierarchical Federated Learning in Wireless Edge Networks

A Joint Learning and Communications Framework for Federated Learning over Wireless Networks

Convergence Analysis and Optimization of Over-the-Air Federated Meta-Learning.

Joint Optimization of Resource Allocation and Data Selection for Fast and Cost-Efficient Federated Edge Learning

Depersonalized Federated Learning: Tackling Statistical Heterogeneity by Alternating Stochastic Gradient Descent