Priorities in indoor environmental science and health, as students see them.

Ellison Carter,C. M. Earnest,Elliott T. Gall,Priscilla Guerrero,D. Hun,M. Jackson,James Lo,B. Stephens,E. Walsh

DOI: https://doi.org/10.1111/j.1600-0668.2009.00632.x

2009-12-01

Abstract:

What problem does this paper attempt to address?

A Joint Communication and Federated Learning Framework for Internet of Things Networks

Zhaohui Yang,Guangyu Jia,Mingzhe Chen,Hak-Keung Lam,Shuguang Cui,H. Vincent Poor,Kai-Kit Wong

DOI: https://doi.org/10.1109/globecom46510.2021.9685679

2021-01-01

Abstract:Federated learning (FL) is widely used in privacy sensitive applications for isolated data islands, with the aim of achieving distributed model training, privacy enhancement and model sharing. Electromyographic (EMG) signals are a type of data collected from wearable sensors of subjects which are distributed on multiple devices, highly personalized and play an important role in several applications including prosthetic hand control, sign languages, grasp recognition, etc. This paper utilizes the FL method to detect single and combined finger movements based on EMG signals. The existing research on FL for wearable healthcare faces challenges of variable probability distributions of data, the need for prerequisite knowledge of server model and computational burdens in parameter transmission. To address these problems, this paper proposes a communication efficient FL framework in which each device only needs to transmit the weight matrices of local models to the server for model aggregation. To further reduce the FL transmission delay, a joint learning and resource allocation problem is formulated via optimizing transmit power of each device, time allocation, and user selection. To solve the delay minimization problem, the objective function is first converted to a tractable expression and then the difference of two convex functions programming is adopted. Simulation results using real EMG signals show that the proposed FL framework with personalized training process successfully detects single and combined finger movements for distributed users. Two public EMG datasets with 10 and 15 different finger movements are employed. Over 98% overall test accuracy is achieved in both datasets which surpasses the conventional learning framework by 1.6% and 0.5% on average. Different scenarios with regard to access points and users are investigated and the convexity of the proposed model is discussed.
Federated Learning over Wireless IoT Networks with Optimized Communication and Resources

Hao Chen,Shaocheng Huang,Deyou Zhang,Ming Xiao,Mikael Skoglund,H. Vincent Poor

DOI: https://doi.org/10.1109/jiot.2022.3151193

IF: 10.6

2022-01-01

IEEE Internet of Things Journal

Abstract:To leverage massive distributed data and computation resources, machine learning in the network edge is considered to be a promising technique, especially for large-scale model training. Federated learning (FL), as a paradigm of collaborative learning techniques, has obtained increasing research attention with the benefits of communication efficiency and improved data privacy. Due to the lossy communication channels and limited communication resources (e.g., bandwidth and power), it is of interest to investigate fast responding and accurate FL schemes over wireless systems. Hence, we investigate the problem of jointly optimized communication efficiency and resources for FL over wireless Internet of Things (IoT) networks. To reduce complexity, we divide the overall optimization problem into two subproblems, i.e., the client scheduling problem and the resource allocation problem. To reduce the communication costs for FL in wireless IoT networks, a new client scheduling policy is proposed by reusing stale local model parameters. To maximize successful information exchange over networks, a Lagrange multiplier method is first leveraged by decoupling variables, including power variables, bandwidth variables, and transmission indicators. Then, a linear-search-based power and bandwidth allocation method is developed. Given appropriate hyperparameters, we show that the proposed communication-efficient FL (CEFL) framework converges at a strong linear rate. Through extensive experiments, it is revealed that the proposed CEFL framework substantially boosts both the communication efficiency and learning performance of both training loss and test accuracy for FL over wireless IoT networks compared to a basic FL approach with uniform resource allocation.

computer science, information systems,telecommunications,engineering, electrical & electronic
Online-Score-Aided Federated Learning: Taming the Resource Constraints in Wireless Networks

Md Ferdous Pervej,Minseok Choi,Andreas F. Molisch

2024-09-01

Abstract:While FL is a widely popular distributed ML strategy that protects data privacy, time-varying wireless network parameters and heterogeneous system configurations of the wireless device pose significant challenges. Although the limited radio and computational resources of the network and the clients, respectively, are widely acknowledged, two critical yet often ignored aspects are (a) wireless devices can only dedicate a small chunk of their limited storage for the FL task and (b) new training samples may arrive in an online manner in many practical wireless applications. Therefore, we propose a new FL algorithm called OSAFL, specifically designed to learn tasks relevant to wireless applications under these practical considerations. Since it has long been proven that under extreme resource constraints, clients may perform an arbitrary number of local training steps, which may lead to client drift under statistically heterogeneous data distributions, we leverage normalized gradient similarities and exploit weighting clients' updates based on optimized scores that facilitate the convergence rate of the proposed OSAFL algorithm. Our extensive simulation results on two different tasks -- each with three different datasets -- with four popular ML models validate the effectiveness of OSAFL compared to six existing state-of-the-art FL baselines.

Machine Learning,Distributed, Parallel, and Cluster Computing,Networking and Internet Architecture,Systems and Control
Hierarchical Federated Learning with Adaptive Clustering on Non-IID Data

Yuqing Tian,Zhaoyang Zhang,Zhaohui Yang,Richeng Jin

DOI: https://doi.org/10.1109/globecom48099.2022.10000749

2022-01-01

Abstract:Federated learning (FL) in a mobile edge network faces challenges from both communication and learning perspectives. The typically non-i.i.d. data can lead to slow convergence and low accuracy. To ease these challenges, frequent communications between user equipments (UEs) and the central macro base station (MBS) are necessary, aggravating the communication burden. In this paper, a novel hierarchical FL framework is proposed to alleviate the biased convergence of the global model, achieving better communication and computation efficiency. Specifically, the UEs are adaptively clustered and allocated to specific small base stations (SBSs) according to channel conditions, geographic locations, and data distributions. The SBSs are further aggregated to the MBS, forming a hierarchical FL framework. The joint user clustering and wireless resource allocation optimization problem is formulated. To solve this problem, a cross entropy (CE) based method with low computational complexity is proposed. Simulation results validate that the proposed hierarchical FL system can save more than 87 percent training time under the EMNIST Letters dataset, achieving fast convergence and significantly improving the system efficiency.
Federated Learning in Multi-RIS-Aided Systems

Wanli Ni,Yuanwei Liu,Zhaohui Yang,Hui Tian,Xuemin Shen

DOI: https://doi.org/10.1109/jiot.2021.3130444

IF: 10.6

2022-01-01

IEEE Internet of Things Journal

Abstract:The fundamental communication paradigms in the next-generation mobile networks are shifting from connected things to connected intelligence. The potential result is that current communication-centric wireless systems are greatly stressed when supporting computation-centric intelligent services with distributed big data. This is one reason that makes federated learning come into being, it allows collaborative training over many edge devices while avoiding the transmission of raw data. To tackle the problem of model aggregation in federated learning systems, this article resorts to multiple reconfigurable intelligent surfaces (RISs) to achieve efficient and reliable learning-oriented wireless connectivity. The seamless integration of communication and computation is actualized by over-the-air computation (AirComp), which can be deemed as one of the uplink nonorthogonal multiple access (NOMA) techniques without individual information decoding. Since all local parameters are uploaded via noisy concurrent transmissions, the unfavorable propagation error inevitably deteriorates the accuracy of the aggregated global model. The goals of this work are to 1) alleviate the signal distortion of AirComp over shared wireless channels and 2) speed up the convergence rate of federated learning. More specifically, both the mean-square error (MSE) and the device set in the model uploading process are optimized by jointly designing transceivers, tuning reflection coefficients, and selecting clients. Compared to baselines, extensive simulation results show that 1) the proposed algorithms can aggregate model more accurately and accelerate convergence and 2) the training loss and inference accuracy of federated learning can be improved significantly with the aid of multiple RISs.
Client Selection for Federated Learning With Non-IID Data in Mobile Edge Computing

Wenyu Zhang,Xiumin Wang,Pan Zhou,Weiwei Wu,Xinglin Zhang

DOI: https://doi.org/10.1109/access.2021.3056919

IF: 3.9

2021-01-01

IEEE Access

Abstract:Federated Learning (FL) has recently attracted considerable attention in internet of things, due to its capability of enabling mobile clients to collaboratively learn a global prediction model without sharing their privacy-sensitive data to the server. Despite its great potential, a main challenge of FL is that the training data are usually non-Independent, Identically Distributed (non-IID) on the clients, which may bring the biases in the model training and cause possible accuracy degradation. To address this issue, this paper aims to propose a novel FL algorithm to alleviate the accuracy degradation caused by non-IID data at clients. Firstly, we observe that the clients with different degrees of non-IID data present heterogeneous weight divergence with the clients owning IID data. Inspired by this, we utilize weight divergence to recognize the non-IID degrees of clients. Then, we propose an efficient FL algorithm, named CSFedAvg, in which the clients with lower degree of non-IID data will be chosen to train the models with higher frequency. Finally, we conduct simulations using publicly-available datasets to train deep neural networks. Simulation results show that the proposed FL algorithm improves the training performance compared with existing FL protocol.

computer science, information systems,telecommunications,engineering, electrical & electronic
Efficiency-Boosting Federated Learning in Wireless Networks: A Long-Term Perspective

Yun Ji,Xiaoxiong Zhong,Zhoubin Kou,Sheng Zhang,Hangfan Li,Yuanyuan Yang

DOI: https://doi.org/10.1109/tvt.2023.3250273

IF: 6.8

2023-01-01

IEEE Transactions on Vehicular Technology

Abstract:Federated learning (FL) can train a global model from clients' local dataset, which can make full use of the computing resources of clients and performs more extensive and efficient machine learning applications on clients with protecting user information requirements. Many existing works have focused on optimizing FL accuracy within the resource constrained in each individual round. However there are few works comprehensively consider the optimization for latency, accuracy and energy consumption from a long-term perspective in wireless federated learning. Inspired by this, in this paper, we investigate FL in wireless networks where client selection and bandwidth allocation are two crucial factors for improving the FL accuracy as well as reducing latency and energy consumption. We formulate the optimization problem as a mixed-integer problem, which minimizes the cost function over finite rounds while satisfying energy budget constraints of each client from a long-term perspective. To address this optimization problem, we propose the Per-round Energy Drift Plus Cost (PEDPC) algorithm in an online manner, including two parts: client selection and bandwidth allocation, which can be addressed by Increasing Time-Maximum Client Selection (ITMCS) algorithm and Barrier Method, respectively. Finally, the performance of the PEDPC algorithm is verified by extensive simulations in terms of latency, accuracy and energy consumption in IID and NON-IID data distributions.
Communication-Efficient Personalized Federated Learning for Green Communications in IoMT

Ziqi Chen,Jun Du,Chunxiao Jiang,Yunlong Lu,Zhu Han

DOI: https://doi.org/10.1109/icc51166.2024.10622427

2024-01-01

Abstract:The rapid development of the Internet of Medical Things (IoMT) has brought about an enormous amount of healthcare data. Effectively and securely processing this sensitive data has become a significant challenge for green communication and privacy protection of the IoMT. As a decentralized learning framework, Federate learning (FL) enables model training without directly aggregating users' raw data, thus ensuring user privacy protection. Moreover, numerous studies have put forth various approaches to enhance the efficiency of FL by minimizing communication costs, yet they may not fully account for the unique characteristics of IoMT. Specifically, the efficiency and performance of model training are closely related to patient life and health. Meanwhile, existing research has indicated that reducing communication costs can result in a decline in training accuracy, which may be critical to patient health. Therefore, aimed at green communication and ensuring the model accuracy, we design a communication-efficient personalized federated learning framework, namely pFedCAS. Specifically, we introduce a control unit, which enables adaptive sparsity of local models, to reduce training costs. Furthermore, a selection unit based on communication quality is added into the global aggregation, which can select suitable clients for model updating. Simulation results validate that the proposed method can significantly reduce communication costs while ensuring the model accuracy. Additionally, The simulation results also validate the excellent robustness of our method to non-iid healthcare data.
IRS Assisted Federated Learning A Broadband Over-the-Air Aggregation Approach

Deyou Zhang,Ming Xiao,Zhibo Pang,Lihui Wang,H. Vincent Poor

2023-10-11

Abstract:We consider a broadband over-the-air computation empowered model aggregation approach for wireless federated learning (FL) systems and propose to leverage an intelligent reflecting surface (IRS) to combat wireless fading and noise. We first investigate the conventional node-selection based framework, where a few edge nodes are dropped in model aggregation to control the aggregation error. We analyze the performance of this node-selection based framework and derive an upper bound on its performance loss, which is shown to be related to the selected edge nodes. Then, we seek to minimize the mean-squared error (MSE) between the desired global gradient parameters and the actually received ones by optimizing the selected edge nodes, their transmit equalization coefficients, the IRS phase shifts, and the receive factors of the cloud server. By resorting to the matrix lifting technique and difference-of-convex programming, we successfully transform the formulated optimization problem into a convex one and solve it using off-the-shelf solvers. To improve learning performance, we further propose a weight-selection based FL framework. In such a framework, we assign each edge node a proper weight coefficient in model aggregation instead of discarding any of them to reduce the aggregation error, i.e., amplitude alignment of the received local gradient parameters from different edge nodes is not required. We also analyze the performance of this weight-selection based framework and derive an upper bound on its performance loss, followed by minimizing the MSE via optimizing the weight coefficients of the edge nodes, their transmit equalization coefficients, the IRS phase shifts, and the receive factors of the cloud server. Furthermore, we use the MNIST dataset for simulations to evaluate the performance of both node-selection and weight-selection based FL frameworks.

Information Theory,Signal Processing
IRS Assisted Federated Learning: A Broadband Over-the-Air Aggregation Approach

Deyou Zhang,Ming Xiao,Zhibo Pang,Lihui Wang,H. Vincent Poor

DOI: https://doi.org/10.1109/twc.2023.3313968

IF: 10.4

2024-01-01

IEEE Transactions on Wireless Communications

Abstract:We consider a broadband over-the-air computation empowered model aggregation approach for wireless federated learning (FL) systems and propose to leverage an intelligent reflecting surface (IRS) to combat wireless fading and noise. We first investigate the conventional node-selection based framework, where a few edge nodes are dropped in model aggregation to control the aggregation error. We analyze the performance of this node-selection based framework and derive an upper bound on its performance loss, which is shown to be related to the selected edge nodes. Then, we seek to minimize the mean-squared error (MSE) between the desired global gradient parameters and the actually received ones by optimizing the selected edge nodes, their transmit equalization coefficients, the IRS phase shifts, and the receive factors of the cloud server. By resorting to the matrix lifting technique and difference-of-convex programming, we successfully transform the formulated optimization problem into a convex one and solve it using off-the-shelf solvers. To improve learning performance, we further propose a weight-selection based FL framework. In such a framework, we assign each edge node a proper weight coefficient in model aggregation instead of discarding any of them to reduce the aggregation error, i.e., amplitude alignment of the received local gradient parameters from different edge nodes is not required.We also analyze the performance of this weight-selection based framework and derive an upper bound on its performance loss, followed by minimizing the MSE via optimizing the weight coefficients of the edge nodes, their transmit equalization coefficients, the IRS phase shifts, and the receive factors of the cloud server. Furthermore, we use the MNIST dataset for simulations to evaluate the performance of both node-selection and weight-selection based FL frameworks.

telecommunications,engineering, electrical & electronic
FLCC: Efficient Distributed Federated Learning on IoMT over CSMA/CA

Abdelaziz Salama,Syed Ali Zaidi,Des McLernon,Mohammed M. H. Qazzaz

2023-03-30

Abstract:Federated Learning (FL) has emerged as a promising approach for privacy preservation, allowing sharing of the model parameters between users and the cloud server rather than the raw local data. FL approaches have been adopted as a cornerstone of distributed machine learning (ML) to solve several complex use cases. FL presents an interesting interplay between communication and ML performance when implemented over distributed wireless nodes. Both the dynamics of networking and learning play an important role. In this article, we investigate the performance of FL on an application that might be used to improve a remote healthcare system over ad hoc networks which employ CSMA/CA to schedule its transmissions. Our FL over CSMA/CA (FLCC) model is designed to eliminate untrusted devices and harness frequency reuse and spatial clustering techniques to improve the throughput required for coordinating a distributed implementation of FL in the wireless network. In our proposed model, frequency allocation is performed on the basis of spatial clustering performed using virtual cells. Each cell assigns a FL server and dedicated carrier frequencies to exchange the updated model's parameters within the cell. We present two metrics to evaluate the network performance: 1) probability of successful transmission while minimizing the interference, and 2) performance of distributed FL model in terms of accuracy and loss while considering the networking dynamics. We benchmark the proposed approach using a well-known MNIST dataset for performance evaluation. We demonstrate that the proposed approach outperforms the baseline FL algorithms in terms of explicitly defining the chosen users' criteria and achieving high accuracy in a robust network.

Distributed, Parallel, and Cluster Computing,Machine Learning,Systems and Control
Self-Balancing Federated Learning With Global Imbalanced Data in Mobile Systems

Moming Duan,Duo Liu,Xianzhang Chen,Renping Liu,Yujuan Tan,Liang Liang

DOI: https://doi.org/10.1109/tpds.2020.3009406

IF: 5.3

2021-01-01

IEEE Transactions on Parallel and Distributed Systems

Abstract:Federated learning (FL) is a distributed deep learning method that enables multiple participants, such as mobile and IoT devices, to contribute a neural network while their private training data remains in local devices. This distributed approach is promising in the mobile systems where have a large corpus of decentralized data and require high privacy. However, unlike the common datasets, the data distribution of the mobile systems is imbalanced which will increase the bias of model. In this article, we demonstrate that the imbalanced distributed training data will cause an accuracy degradation of FL applications. To counter this problem, we build a self-balancing FL framework named Astraea, which alleviates the imbalances by 1) Z-score-based data augmentation, and 2) Mediator-based multi-client rescheduling. The proposed framework relieves global imbalance by adaptive data augmentation and downsampling, and for averaging the local imbalance, it creates the mediator to reschedule the training of clients based on Kullback–Leibler divergence (KLD) of their data distribution. Compared with FedAvg, the vanilla FL algorithm, Astraea shows +4.39 and +6.51 percent improvement of top-1 accuracy on the imbalanced EMNIST and imbalanced CINIC-10 datasets, respectively. Meanwhile, the communication traffic of Astraea is reduced by 75 percent compared to FedAvg.

computer science, theory & methods,engineering, electrical & electronic
Client Selection and Bandwidth Allocation in Wireless Federated Learning Networks: A Long-Term Perspective

Jie Xu,Heqiang Wang

DOI: https://doi.org/10.1109/twc.2020.3031503

IF: 10.4

2021-02-01

IEEE Transactions on Wireless Communications

Abstract:This paper studies federated learning (FL) in a classic wireless network, where learning clients share a common wireless link to a coordinating server to perform federated model training using their local data. In such wireless federated learning networks (WFLNs), optimizing the learning performance depends crucially on how clients are selected and how bandwidth is allocated among the selected clients in every learning round, as both radio and client energy resources are limited. While existing works have made some attempts to allocate the limited wireless resources to optimize FL, they focus on the problem in individual learning rounds, overlooking an inherent yet critical feature of federated learning. This paper brings a new long-term perspective to resource allocation in WFLNs, realizing that learning rounds are not only temporally interdependent but also have varying significance towards the final learning outcome. To this end, we first design data-driven experiments to show that different temporal client selection patterns lead to considerably different learning performance. With the obtained insights, we formulate a stochastic optimization problem for joint client selection and bandwidth allocation under long-term client energy constraints, and develop a new algorithm that utilizes only currently available wireless channel information but can achieve long-term performance guarantee. Experiments show that our algorithm results in the desired temporal client selection pattern, is adaptive to changing network environments and far outperforms benchmarks that ignore the long-term effect of FL.

telecommunications,engineering, electrical & electronic
Efficient Wireless Federated Learning with Partial Model Aggregation

Zhixiong Chen,Wenqiang Yi,Arumugam Nallanathan,Geoffrey Ye Li

DOI: https://doi.org/10.48550/arXiv.2204.09746

IF: 5.414

2022-04-20

Machine Learning

Abstract:The data heterogeneity across devices and the limited communication resources, e.g., bandwidth and energy, are two of the main bottlenecks for wireless federated learning (FL). To tackle these challenges, we first devise a novel FL framework with partial model aggregation (PMA). This approach aggregates the lower layers of neural networks, responsible for feature extraction, at the parameter server while keeping the upper layers, responsible for complex pattern recognition, at devices for personalization. The proposed PMA-FL is able to address the data heterogeneity and reduce the transmitted information in wireless channels. Then, we derive a convergence bound of the framework under a non-convex loss function setting to reveal the role of unbalanced data size in the learning performance. On this basis, we maximize the scheduled data size to minimize the global loss function through jointly optimize the device scheduling, bandwidth allocation, computation and communication time division policies with the assistance of Lyapunov optimization. Our analysis reveals that the optimal time division is achieved when the communication and computation parts of PMA-FL have the same power. We also develop a bisection method to solve the optimal bandwidth allocation policy and use the set expansion algorithm to address the device scheduling policy. Compared with the benchmark schemes, the proposed PMA-FL improves 3.13\% and 11.8\% accuracy on two typical datasets with heterogeneous data distribution settings, i.e., MINIST and CIFAR-10, respectively. In addition, the proposed joint dynamic device scheduling and resource management approach achieve slightly higher accuracy than the considered benchmarks, but they provide a satisfactory energy and time reduction: 29\% energy or 20\% time reduction on the MNIST; and 25\% energy or 12.5\% time reduction on the CIFAR-10.
Federated Multi-Task Learning with Non-Stationary and Heterogeneous Data in Wireless Networks

Hongwei Zhang,Meixia Tao,Yuanming Shi,Xiaoyan Bi,Khaled B. Letaief

DOI: https://doi.org/10.1109/twc.2023.3301611

IF: 10.4

2023-01-01

IEEE Transactions on Wireless Communications

Abstract:Federated multi-task learning (FMTL) is a promising edge learning framework to fit the data with non-independent and non-identical distribution (non-i.i.d.) by leveraging the statistical correlations among the personalized models. For many practical applications in wireless communications, the sensory data are not only heterogeneous but also non-stationary due to the mobility of terminals and the randomness of link connections. The non-stationary heterogeneous data may lead to model divergence and staleness in the training stage and poor test accuracy in the inference stage. In this paper, we shall develop an adaptive FMTL framework, which works well with non-stationary data. We further propose to optimize the model updating and cluster splitting schemes in the training stage to accelerate model convergence. We also design a low-complexity model selection and pruning schemes in both the training and inference stages to select the best model for fitting the current data and delete redundant models, respectively. The proposed framework is validated in the edge learning model, namely, the linear regression problem for indoor localization in wireless networks and GNN for wireless power control problems. Numerical results demonstrate that the proposed framework can accelerate the model training convergence and reduce the computation complexity while ensuring model accuracy.

telecommunications,engineering, electrical & electronic
Risk-Aware Accelerated Wireless Federated Learning with Heterogeneous Clients

Mohamed Ads,Hesham ElSawy,Hossam S. Hassanein

2024-01-17

Abstract:Wireless Federated Learning (FL) is an emerging distributed machine learning paradigm, particularly gaining momentum in domains with confidential and private data on mobile clients. However, the location-dependent performance, in terms of transmission rates and susceptibility to transmission errors, poses major challenges for wireless FL's convergence speed and accuracy. The challenge is more acute for hostile environments without a metric that authenticates the data quality and security profile of the clients. In this context, this paper proposes a novel risk-aware accelerated FL framework that accounts for the clients heterogeneity in the amount of possessed data, transmission rates, transmission errors, and trustworthiness. Classifying clients according to their location-dependent performance and trustworthiness profiles, we propose a dynamic risk-aware global model aggregation scheme that allows clients to participate in descending order of their transmission rates and an ascending trustworthiness constraint. In particular, the transmission rate is the dominant participation criterion for initial rounds to accelerate the convergence speed. Our model then progressively relaxes the transmission rate restriction to explore more training data at cell-edge clients. The aggregation rounds incorporate a debiasing factor that accounts for transmission errors. Risk-awareness is enabled by a validation set, where the base station eliminates non-trustworthy clients at the fine-tuning stage. The proposed scheme is benchmarked against a conservative scheme (i.e., only allowing trustworthy devices) and an aggressive scheme (i.e., oblivious to the trust metric). The numerical results highlight the superiority of the proposed scheme in terms of accuracy and convergence speed when compared to both benchmarks.

Machine Learning
Federated Learning with Additional Mechanisms on Clients to Reduce Communication Costs

Yao Xin,Huang Tianchi,Wu Chenglei,Zhang Rui-xiao,Sun Lifeng

2019-01-01

Abstract: Federated learning (FL) enables on-device training over distributed networks consisting of a massive amount of modern smart devices, such as smartphones and IoT devices. However, the leading optimization algorithm in such settings, i.e., federated averaging (FedAvg), suffers from heavy communication cost and inevitable performance drop, especially when the local data is distributed in a non-IID way. To alleviate this problem, we propose two potential solutions by introducing additional mechanisms to the on-device training. The first (FedMMD) is adopting a two-stream model with the MMD (Maximum Mean Discrepancy) constraint instead of a single model in vanilla FedAvg to be trained on devices. Experiments show that the proposed method outperforms baselines, especially in non-IID FL settings, with a reduction of more than 20% in required communication rounds. The second is FL with feature fusion (FedFusion). By aggregating the features from both the local and global models, we achieve higher accuracy at less communication cost. Furthermore, the feature fusion modules offer better initialization for newly incoming clients and thus speed up the process of convergence. Experiments in popular FL scenarios show that our FedFusion outperforms baselines in both accuracy and generalization ability while reducing the number of required communication rounds by more than 60%.
Latency Aware Semi-synchronous Client Selection and Model Aggregation for Wireless Federated Learning

Liangkun Yu,Xiang Sun,Rana Albelaihi,Chen Yi

DOI: https://doi.org/10.48550/arXiv.2210.10311

2022-11-29

Abstract:Federated learning (FL) is a collaborative machine learning framework that requires different clients (e.g., Internet of Things devices) to participate in the machine learning model training process by training and uploading their local models to an FL server in each global iteration. Upon receiving the local models from all the clients, the FL server generates a global model by aggregating the received local models. This traditional FL process may suffer from the straggler problem in heterogeneous client settings, where the FL server has to wait for slow clients to upload their local models in each global iteration, thus increasing the overall training time. One of the solutions is to set up a deadline and only the clients that can upload their local models before the deadline would be selected in the FL process. This solution may lead to a slow convergence rate and global model overfitting issues due to the limited client selection. In this paper, we propose the Latency awarE Semi-synchronous client Selection and mOdel aggregation for federated learNing (LESSON) method that allows all the clients to participate in the whole FL process but with different frequencies. That is, faster clients would be scheduled to upload their models more frequently than slow clients, thus resolving the straggler problem and accelerating the convergence speed, while avoiding model overfitting. Also, LESSON is capable of adjusting the tradeoff between the model accuracy and convergence rate by varying the deadline. Extensive simulations have been conducted to compare the performance of LESSON with the other two baseline methods, i.e., FedAvg and FedCS. The simulation results demonstrate that LESSON achieves faster convergence speed than FedAvg and FedCS, and higher model accuracy than FedCS.

Distributed, Parallel, and Cluster Computing,Artificial Intelligence
Jointly Optimizing Client Selection and Resource Management in Wireless Federated Learning for Internet of Things

Liangkun Yu,Rana Albelaihi,Xiang Sun,Nirwan Ansari,Michael Devetsikiotis

DOI: https://doi.org/10.1109/jiot.2021.3103715

IF: 10.6

2022-03-15

IEEE Internet of Things Journal

Abstract:Federated learning (FL) has been proposed to efficiently and privacy-preserving distributed machine learning architecture for the Internet of Things (IoT). In a wireless FL system, clients in IoT devices train their local models over the local data sets. The derived local models are uploaded to an FL server to generate a global model, broadcasted to the clients in the next global iteration for further training. Owing to the heterogeneous feature of the clients, client selection is critical to determine the overall training time. Traditionally, the objective of client selection is to select the maximum number of clients who can derive and upload their local models before the deadline in each global iteration. However, selecting more clients increases the energy consumption of the clients. Moreover, selecting the maximum number of clients is unnecessary as having fewer clients in early global iterations and more clients in later global iterations have been proved to achieve higher model accuracy. Hence, this article proposes to dynamically adjust and optimize the tradeoff between maximizing the number of selected clients and minimizing the total energy consumption of the clients by selecting suitable clients and allocating appropriate resources in terms of CPU frequency and transmission power. We formulate the joint client selection and resource management problem and design the energy and latency-aware resource management and client selection (ELASTIC) algorithm to efficiently solve the problem. Extensive simulations are conducted to demonstrate the performance of ELASTIC.

computer science, information systems,telecommunications,engineering, electrical & electronic
Adaptive Idle Model Fusion in Hierarchical Federated Learning for Unbalanced Edge Regions

Jiuyun Xu,Hanfei Fan,Qiqi Wang,Yinyue Jiang,Qiang Duan

DOI: https://doi.org/10.1109/tnse.2024.3410275

IF: 6.6

2024-08-18

IEEE Transactions on Network Science and Engineering

Abstract:Federated learning (FL) has gained attention due to the exponential growth of data. However, the Non-IID nature of local data introduces bias into the global model during training. In mobile IoT scenarios, client mobility can lead to data over-aggregation in the edge region, increasing bias in data distribution among edge regions and affecting global model accuracy. Existing solutions mitigate Non-IID problems but overlook the potential contributions of unselected clients with data capable of offsetting imbalances in edge regions. To fill this gap, we first demonstrate that the unbalanced data distribution in the edge regions is one reason for the degraded accuracy of the global model. Then, we propose a client adaptive assignment hierarchical federated learning framework (CAHFL). CAHFL can quantify clients' contribution by calculating Earth Movement Distance (EMD) between clients and servers and rationally selects idle clients to participate in training. In addition, based on the client's contribution, this paper designs an adaptive idle model feature fusion mechanism to enhance the edge model by adaptively fusing the local features of the idle model. Finally, we have performed simulations using publicly available datasets, and the simulation results indicate that the proposed FL framework improves the training performance compared to existing FL protocols.

engineering, multidisciplinary,mathematics, interdisciplinary applications

Priorities in indoor environmental science and health, as students see them.

A Joint Communication and Federated Learning Framework for Internet of Things Networks

Federated Learning over Wireless IoT Networks with Optimized Communication and Resources

Online-Score-Aided Federated Learning: Taming the Resource Constraints in Wireless Networks

Hierarchical Federated Learning with Adaptive Clustering on Non-IID Data

Federated Learning in Multi-RIS-Aided Systems

Client Selection for Federated Learning With Non-IID Data in Mobile Edge Computing

Efficiency-Boosting Federated Learning in Wireless Networks: A Long-Term Perspective

Communication-Efficient Personalized Federated Learning for Green Communications in IoMT

IRS Assisted Federated Learning A Broadband Over-the-Air Aggregation Approach

IRS Assisted Federated Learning: A Broadband Over-the-Air Aggregation Approach

FLCC: Efficient Distributed Federated Learning on IoMT over CSMA/CA

Self-Balancing Federated Learning With Global Imbalanced Data in Mobile Systems

Client Selection and Bandwidth Allocation in Wireless Federated Learning Networks: A Long-Term Perspective

Efficient Wireless Federated Learning with Partial Model Aggregation

Federated Multi-Task Learning with Non-Stationary and Heterogeneous Data in Wireless Networks

Risk-Aware Accelerated Wireless Federated Learning with Heterogeneous Clients

Federated Learning with Additional Mechanisms on Clients to Reduce Communication Costs

Latency Aware Semi-synchronous Client Selection and Model Aggregation for Wireless Federated Learning

Jointly Optimizing Client Selection and Resource Management in Wireless Federated Learning for Internet of Things

Adaptive Idle Model Fusion in Hierarchical Federated Learning for Unbalanced Edge Regions