Abstract:Efficient collaboration between collaborative machine learning and wireless communication technology, forming a Federated Edge Learning (FEEL), has spawned a series of next-generation intelligent applications. However, due to the openness of network connections, the FEEL framework generally involves hundreds of remote devices (or clients), resulting in expensive communication costs, which is not friendly to resource-constrained FEEL. To address this issue, we propose a distributed approximate Newton-type algorithm with fast convergence speed to alleviate the problem of FEEL resource (in terms of communication resources) constraints. Specifically, the proposed algorithm is improved based on distributed L-BFGS algorithm and allows each client to approximate the high-cost Hessian matrix by computing the low-cost Fisher matrix in a distributed manner to find a better descent direction, thereby speeding up convergence. Second, we prove that the proposed algorithm has linear convergence in strongly convex and non-convex cases and analyze its computational and communication complexity. Similarly, due to the heterogeneity of the connected remote devices, FEEL faces the challenge of heterogeneous data and non-IID (Independent and Identically Distributed) data. To this end, we design a simple but elegant training scheme, namely FedOVA (Federated One-vs-All), to solve the heterogeneous statistical challenge brought by heterogeneous data. In this way, FedOVA first decomposes a multi-class classification problem into more straightforward binary classification problems and then combines their respective outputs using ensemble learning. In particular, the scheme can be well integrated with our communication efficient algorithm to serve FEEL. Numerical results verify the effectiveness and superiority of the proposed algorithm.

Resource-Efficient and Convergence-Preserving Online Participant Selection in Federated Learning

Optimize Scheduling of Federated Learning on Battery-powered Mobile Devices

Energy-efficient Federated Learning Via Stabilization-aware On-device Update Scaling.

Theoretical Convergence Guaranteed Resource-Adaptive Federated Learning with Mixed Heterogeneity

Dynamic and Fast Convergence for Federated Learning Via Optimized Hyperparameters

Experience-Driven Computational Resource Allocation of Federated Learning by Deep Reinforcement Learning

Scheduling In-Band Network Telemetry with Convergence-Preserving Federated Learning

Learning for Learning: Predictive Online Control of Federated Learning with Edge Provisioning

Resource-Aware Asynchronous Online Federated Learning for Nonlinear Regression

Joint Optimization of Resource Allocation and Data Selection for Fast and Cost-Efficient Federated Edge Learning

An Online Learning Approach for Client Selection in Federated Edge Learning under Budget Constraint

Online Client Scheduling and Resource Allocation for Efficient Federated Edge Learning

Client Selection and Bandwidth Allocation for Federated Learning: An Online Optimization Perspective

Accelerating Hybrid Federated Learning Convergence under Partial Participation

Communication-Efficient Device Scheduling for Federated Learning Using Stochastic Optimization

Federated Learning with Pareto Optimality for Resource Efficiency and Fast Model Convergence in Mobile Environments

Resource-constrained Federated Learning with Heterogeneous Data: Formulation and Analysis

Robust Federated Learning with Noisy Communication.

Efficient Adaptive Federated Learning in Resource-Constrained IoT Environments

On the Convergence of Heterogeneous Federated Learning with Arbitrary Adaptive Online Model Pruning

Towards Efficient Federated Learning Framework Via Selective Aggregation of Models