Abstract:Federated Learning (FL) has become a viable technique for realizing privacy-enhancing distributed deep learning on the network edge. Heterogeneous hardware, unreliable client devices, and energy constraints often characterize edge computing systems. In this paper, we propose FLEdge, which complements existing FL benchmarks by enabling a systematic evaluation of client capabilities. We focus on computational and communication bottlenecks, client behavior, and data security implications. Our experiments with models varying from 14K to 80M trainable parameters are carried out on dedicated hardware with emulated network characteristics and client behavior. We find that state-of-the-art embedded hardware has significant memory bottlenecks, leading to 4x longer processing times than on modern data center GPUs.

What problem does this paper attempt to address?

The main problem that this paper attempts to solve is to evaluate and improve the application of Federated Learning (FL) in edge - computing systems. Specifically, the authors hope to systematically study the performance of FL workloads in edge - computing environments by introducing the FLEdge benchmarking framework, with particular attention to the following aspects: 1. **Client Behavior**: Research the impact of different types of client behaviors (such as dropout rate, reliability, etc.) on the FL training process. 2. **Communication Efficiency**: Evaluate the feasibility of FL workloads deployed in edge - computing systems under network conditions (such as bandwidth, latency, etc.). 3. **Energy Efficiency**: Analyze the energy consumption of different types of hardware when running FL tasks to optimize resource utilization. 4. **Hardware Diversity**: Consider the heterogeneity of edge devices and evaluate the impact of different hardware configurations on FL performance. ### Detailed Explanation #### 1. Client Behavior In edge - computing environments, the behavior of client devices is often unreliable and may drop out or malfunction frequently. This will have a significant impact on the training effect of FL. Therefore, the paper proposes a method to simulate and evaluate the impact of different client behavior patterns on FL performance. For example, an independent binomial distribution is used to model the probability of client dropout: \[ p_d^m = P(m)=\text{Bin}(p) \] where \(p\) represents the dropout probability, ranging from 0% to 50%. In this way, the impact of different dropout rates on model aggregation can be studied. #### 2. Communication Efficiency Communication efficiency is one of the key factors in FL system design. The paper introduces an indicator called "Granularity" to measure the ratio of communication to computation time: \[ G = \frac{T_{\text{computation}}}{T_{\text{communication}}} \] When \(G\gg1\), it indicates that the distributed workload is advantageous; when \(G\simeq1\) or \(G < 1\), it shows that the effect of distributed processing is not obvious. In addition, the paper also considers the cost - per - bit communication model to evaluate the scalability and cost - effectiveness of large - scale FL systems. #### 3. Energy Efficiency Edge - computing systems are usually subject to energy limitations, especially on remote or mobile devices. The paper evaluates the energy efficiency of different hardware platforms by measuring the real - time energy consumption of embedded devices and calculating the Energy Efficiency Ratio (EER): \[ \eta_e=\frac{Q}{W} \] where \(Q\) represents the throughput (the number of samples processed per unit time), and \(W\) represents the average power consumption. #### 4. Hardware Diversity There are a wide variety of hardware in edge - computing environments with large performance differences. The paper studies the impact of different types of hardware on FL training performance through micro - benchmarking, especially focusing on each stage in the deep - learning steps (such as batch - loading, forward - propagation, loss - calculation, backward - propagation, and optimizer steps). This helps to identify potential performance bottlenecks and provides a basis for optimizing FL workloads. ### Summary FLEdge aims to promote the practical application of federated learning in edge - computing systems by systematically evaluating client behavior, communication efficiency, energy efficiency, and hardware diversity. The research results of the paper not only provide valuable references for researchers but also offer practical guidance for the industry to better deploy and optimize FL systems.

FLEdge: Benchmarking Federated Machine Learning Applications in Edge Computing Systems

Edge-cloud Collaborative Learning with Federated and Centralized Features

Edge AIBench: Towards Comprehensive End-to-End Edge Computing Benchmarking.

Experimental Evaluation and Analysis of Federated Learning in Edge Computing Environments

Federated Fine-Tuning of LLMs on the Very Edge: The Good, the Bad, the Ugly

A Framework for testing Federated Learning algorithms using an edge-like environment

An Efficient Asynchronous Federated Learning Protocol for Edge Devices

An Empirical Study of Federated Learning on IoT-Edge Devices: Resource Allocation and Heterogeneity

Accelerating Decentralized Federated Learning in Heterogeneous Edge Computing

Flower: A Friendly Federated Learning Research Framework

FlocOff: Data Heterogeneity Resilient Federated Learning with Communication-Efficient Edge Offloading

Multi-Edge Server-Assisted Dynamic Federated Learning With an Optimized Floating Aggregation Point

Semi-Decentralized Federated Edge Learning with Data and Device Heterogeneity

FLight: A Lightweight Federated Learning Framework in Edge and Fog Computing

Data-Centric Client Selection for Federated Learning Over Distributed Edge Networks

Online Client Scheduling and Resource Allocation for Efficient Federated Edge Learning

Decentralized Federated Learning on the Edge over Wireless Mesh Networks

PFLF: Privacy-Preserving Federated Learning Framework for Edge Computing

Federated Edge Learning: Design Issues and Challenges

AnycostFL: Efficient On-Demand Federated Learning over Heterogeneous Edge Devices