Abstract:In this work, a <italic>Hybrid Hierarchical Federated Edge Learning</italic> (HHFEL) architecture that consists of a device layer, an edge layer, and a cloud layer over heterogeneous networks, is investigated for large-scale model training. In such systems, learning efficiency is severely degraded by limited communication resources and device heterogeneity in terms of local data distribution and computation capability, especially for synchronous FL mechanisms where the training of each round should wait for the slowest device. To tackle this issue, asynchronous FL is proposed, which allows the devices with powerful computation and communication capabilities exchanging information with the server more frequently. However, this asynchronous FL framework faces a new challenge of low accuracy caused by the imbalanced local model updating. To overcome the shortage of both synchronous and asynchronous FLs, we propose an enhanced online semi-asynchronous FL mechanism between the edge-device layers, where each device trains its local model with the newly generated data and each edge server aggregates a number of local models based on their arrival order in each round. Particularly, devices with faster training speeds would fully utilize the idle time by training their local models repetitively. Meanwhile, synchronous FL with an edge elastic update strategy is adopted to the cloud-edge layers for personalized information exchange. Considering the continuous data generation feature, we formulate the objective problem as an online <italic>Markov Decision Process</italic> (MDP) to realize efficient communication-and-computing HHFEL via joint device selection and resource allocation. Due to the non-convex and combinatorial problem structure, we develop a hybrid <italic>Deep Q-Network</italic> (DQN) and <italic>Deep Deterministic Policy Gradient</italic> (DDPG) approach with low computational complexity to adapt the device selection and resource allocation strategies. Numerical results show the effectiveness of the proposed mechanism compared with existing benchmarks.

Optimizing Federated Edge Learning on Non-IID Data Via Neural Architecture Search

Hierarchical Federated Learning with Adaptive Clustering on Non-IID Data

An Efficient Asynchronous Federated Learning Protocol for Edge Devices

Importance-Aware Data Selection and Resource Allocation in Federated Edge Learning System.

Enhanced Hybrid Hierarchical Federated Edge Learning Over Heterogeneous Networks

Optimizing Federated Learning on Non-IID Data Using Local Shapley Value.

Peaches: Personalized Federated Learning with Neural Architecture Search in Edge Computing

Towards Non-I.I.D. and Invisible Data with FedNAS: Federated Deep Learning via Neural Architecture Search

Federated Learning Using Sparse-Adaptive Model Selection for Embedded Edge Computing

Real-time Federated Evolutionary Neural Architecture Search

Scalable and Low-Latency Federated Learning with Cooperative Mobile Edge Networking

Asynchronous Online Federated Learning for Edge Devices with Non-IID Data

Neural Architecture Search over Decentralized Data.

Client Selection for Federated Learning With Non-IID Data in Mobile Edge Computing

Toward efficient resource utilization at edge nodes in federated learning

Lead federated neuromorphic learning for wireless edge artificial intelligence

Cost-Efficient Federated Learning for Edge Intelligence in Multi-Cell Networks

Resource-Aware Federated Neural Architecture Search over Heterogeneous Mobile Devices

Adaptive Federated Learning in Resource Constrained Edge Computing Systems

Efficient and Less Centralized Federated Learning

Snowball Effect in Federated Learning: an Approach of Exponentially Expanding Structures for Optimizing the Training Efficiency