Preface to Federated Learning: Algorithms, Systems, and Applications: Part 2

Qiang Yang,Yongxin Tong,Yang Liu,Yangqiu Song,Hao Peng,Boi Faltings

DOI: https://doi.org/10.1145/3536420

IF: 5

2022-08-24

ACM Transactions on Intelligent Systems and Technology

Abstract:No abstract available.

computer science, information systems, artificial intelligence

What problem does this paper attempt to address?

Introduction to the Special Issue on the Federated Learning: Algorithms, Systems, and Applications: Part 1

Qiang Yang,Yongxin Tong,Yang Liu,Yangqiu Song,Hao Peng,Boi Faltings

DOI: https://doi.org/10.1145/3514223

IF: 5

2022-01-01

ACM Transactions on Intelligent Systems and Technology

Abstract:introduction Introduction to the Special Issue on the Federated Learning: Algorithms, Systems, and Applications: Part 1 Editors: Qiang Yang WeBank, Hong Kong University of Science and Technology, China WeBank, Hong Kong University of Science and Technology, ChinaSearch about this author , Yongxin Tong Beihang University, China Beihang University, ChinaSearch about this author , Yang Liu WeBank, China WeBank, ChinaSearch about this author , Yangqiu Song Hong Kong University of Science and Technology, China Hong Kong University of Science and Technology, ChinaSearch about this author , Hao Peng Beihang University, China Beihang University, ChinaSearch about this author , Boi Faltings Ecole Polytechnique Federale de Lausanne, Switzerland Ecole Polytechnique Federale de Lausanne, SwitzerlandSearch about this author Authors Info & Claims ACM Transactions on Intelligent Systems and TechnologyVolume 13Issue 4August 2022 Article No.: 52pp 1–3https://doi.org/10.1145/3514223Published:25 August 2022Publication History 0citation238DownloadsMetricsTotal Citations0Total Downloads238Last 12 Months238Last 6 weeks63 Get Citation AlertsNew Citation Alert added!This alert has been successfully added and will be sent to:You will be notified whenever a record that you have chosen has been cited.To manage your alert preferences, click on the button below.Manage my AlertsNew Citation Alert!Please log in to your account Save to BinderSave to BinderCreate a New BinderNameCancelCreateExport CitationPublisher SiteGet Access
Federated Machine Learning: Concept and Applications

Qiang Yang,Yang Liu,Tianjian Chen,Yongxin Tong

DOI: https://doi.org/10.48550/arXiv.1902.04885

IF: 14.4

2019-02-13

Artificial Intelligence

Abstract:Today's AI still faces two major challenges. One is that in most industries, data exists in the form of isolated islands. The other is the strengthening of data privacy and security. We propose a possible solution to these challenges: secure federated learning. Beyond the federated learning framework first proposed by Google in 2016, we introduce a comprehensive secure federated learning framework, which includes horizontal federated learning, vertical federated learning and federated transfer learning. We provide definitions, architectures and applications for the federated learning framework, and provide a comprehensive survey of existing works on this subject. In addition, we propose building data networks among organizations based on federated mechanisms as an effective solution to allow knowledge to be shared without compromising user privacy.
A Generalized Look at Federated Learning: Survey and Perspectives

Taki Hasan Rafi,Faiza Anan Noor,Tahmid Hussain,Dong-Kyu Chae,Zhaohui Yang

DOI: https://doi.org/10.48550/arxiv.2303.14787

2023-01-01

Abstract:Federated learning (FL) refers to a distributed machine learning framework involving learning from several decentralized edge clients without sharing local dataset. This distributed strategy prevents data leakage and enables on-device training as it updates the global model based on the local model updates. Despite offering several advantages, including data privacy and scalability, FL poses challenges such as statistical and system heterogeneity of data in federated networks, communication bottlenecks, privacy and security issues. This survey contains a systematic summarization of previous work, studies, and experiments on FL and presents a list of possibilities for FL across a range of applications and use cases. Other than that, various challenges of implementing FL and promising directions revolving around the corresponding challenges are provided.
Federated Learning -- Methods, Applications and beyond

Moritz Heusinger,Christoph Raab,Fabrice Rossi,Frank-Michael Schleif

DOI: https://doi.org/10.14428/esann/2021.ES2021-4

2022-12-22

Abstract:In recent years the applications of machine learning models have increased rapidly, due to the large amount of available data and technological <a class="link-external link-http" href="http://progress.While" rel="external noopener nofollow">this http URL</a> some domains like web analysis can benefit from this with only minor restrictions, other fields like in medicine with patient data are strongerregulated. In particular \emph{data privacy} plays an important role as recently highlighted by the trustworthy AI initiative of the EU or general privacy regulations in legislation. Another major challenge is, that the required training \emph{data is} often \emph{distributed} in terms of features or samples and unavailable for classicalbatch learning approaches. In 2016 Google came up with a framework, called \emph{Federated Learning} to solve both of these problems. We provide a brief overview on existing Methods and Applications in the field of vertical and horizontal \emph{Federated Learning}, as well as \emph{Fderated Transfer Learning}.

Machine Learning
Towards Federated Learning at Scale: System Design

Keith Bonawitz,Hubert Eichner,Wolfgang Grieskamp,Dzmitry Huba,Alex Ingerman,Vladimir Ivanov,Chloe Kiddon,Jakub Konečný,Stefano Mazzocchi,H. Brendan McMahan,Timon Van Overveldt,David Petrou,Daniel Ramage,Jason Roselander

DOI: https://doi.org/10.48550/arXiv.1902.01046

IF: 5.414

2019-02-04

Machine Learning

Abstract:Federated Learning is a distributed machine learning approach which enables model training on a large corpus of decentralized data. We have built a scalable production system for Federated Learning in the domain of mobile devices, based on TensorFlow. In this paper, we describe the resulting high-level design, sketch some of the challenges and their solutions, and touch upon the open problems and future directions.
Federated Learning: Challenges, Methods, and Future Directions

Anit Kumar Sahu,Ameet Talwalkar,Tian Li,Virginia Smith

DOI: https://doi.org/10.1109/MSP.2020.2975749

IF: 15.204

2019-08-21

IEEE Signal Processing Magazine

Abstract:Federated learning involves training statistical models over remote devices or siloed data centers, such as mobile phones or hospitals, while keeping data localized. Training in heterogeneous and potentially massive networks introduces novel challenges that require a fundamental departure from standard approaches for large-scale machine learning, distributed optimization, and privacy-preserving data analysis. In this article, we discuss the unique characteristics and challenges of federated learning, provide a broad overview of current approaches, and outline several directions of future work that are relevant to a wide range of research communities.

Computer Science,Mathematics
Federated learning: Overview, strategies, applications, tools and future directions

Betul Yurdem,Murat Kuzlu,Mehmet Kemal Gullu,Ferhat Ozgur Catak,Maliha Tabassum

DOI: https://doi.org/10.1016/j.heliyon.2024.e38137

IF: 3.776

2024-09-20

Heliyon

Abstract:Federated learning (FL) is a distributed machine learning process, which allows multiple nodes to work together to train a shared model without exchanging raw data. It offers several key advantages, such as data privacy, security, efficiency, and scalability, by keeping data local and only exchanging model updates through the communication network. This review paper provides a comprehensive overview of federated learning, including its principles, strategies, applications, and tools along with opportunities, challenges, and future research directions. The findings of this paper emphasize that federated learning strategies can significantly help overcome privacy and confidentiality concerns, particularly for high-risk applications.
Federated Learning in Practice: Reflections and Projections

Katharine Daly,Hubert Eichner,Peter Kairouz,H. Brendan McMahan,Daniel Ramage,Zheng Xu

2024-10-11

Abstract:Federated Learning (FL) is a machine learning technique that enables multiple entities to collaboratively learn a shared model without exchanging their local data. Over the past decade, FL systems have achieved substantial progress, scaling to millions of devices across various learning domains while offering meaningful differential privacy (DP) guarantees. Production systems from organizations like Google, Apple, and Meta demonstrate the real-world applicability of FL. However, key challenges remain, including verifying server-side DP guarantees and coordinating training across heterogeneous devices, limiting broader adoption. Additionally, emerging trends such as large (multi-modal) models and blurred lines between training, inference, and personalization challenge traditional FL frameworks. In response, we propose a redefined FL framework that prioritizes privacy principles rather than rigid definitions. We also chart a path forward by leveraging trusted execution environments and open-source ecosystems to address these challenges and facilitate future advancements in FL.

Machine Learning,Artificial Intelligence,Cryptography and Security
Preconditioned Federated Learning

Zeyi Tao,Jindi Wu,Qun Li

2023-09-20

Abstract:Federated Learning (FL) is a distributed machine learning approach that enables model training in communication efficient and privacy-preserving manner. The standard optimization method in FL is Federated Averaging (FedAvg), which performs multiple local SGD steps between communication rounds. FedAvg has been considered to lack algorithm adaptivity compared to modern first-order adaptive optimizations. In this paper, we propose new communication-efficient FL algortithms based on two adaptive frameworks: local adaptivity (PreFed) and server-side adaptivity (PreFedOp). Proposed methods adopt adaptivity by using a novel covariance matrix preconditioner. Theoretically, we provide convergence guarantees for our algorithms. The empirical experiments show our methods achieve state-of-the-art performances on both i.i.d. and non-i.i.d. settings.

Machine Learning,Artificial Intelligence
Federated Select: A Primitive for Communication- and Memory-Efficient Federated Learning

Zachary Charles,Kallista Bonawitz,Stanislav Chiknavaryan,Brendan McMahan,Blaise Agüera y Arcas

DOI: https://doi.org/10.48550/arXiv.2208.09432

2022-08-20

Abstract:Federated learning (FL) is a framework for machine learning across heterogeneous client devices in a privacy-preserving fashion. To date, most FL algorithms learn a "global" server model across multiple rounds. At each round, the same server model is broadcast to all participating clients, updated locally, and then aggregated across clients. In this work, we propose a more general procedure in which clients "select" what values are sent to them. Notably, this allows clients to operate on smaller, data-dependent slices. In order to make this practical, we outline a primitive, federated select, which enables client-specific selection in realistic FL systems. We discuss how to use federated select for model training and show that it can lead to drastic reductions in communication and client memory usage, potentially enabling the training of models too large to fit on-device. We also discuss the implications of federated select on privacy and trust, which in turn affect possible system constraints and design. Finally, we discuss open questions concerning model architectures, privacy-preserving technologies, and practical FL systems.

Machine Learning,Distributed, Parallel, and Cluster Computing
Federated Learning: Organizational Opportunities, Challenges, and Adoption Strategies

Joaquin Delgado Fernandez,Martin Brennecke,Tom Barbereau,Alexander Rieger,Gilbert Fridgen

2023-09-06

Abstract:Restrictive rules for data sharing in many industries have led to the development of federated learning. Federated learning is a machine-learning technique that allows distributed clients to train models collaboratively without the need to share their respective training data with others. In this paper, we first explore the technical foundations of federated learning and its organizational opportunities. Second, we present a conceptual framework for the adoption of federated learning, mapping four types of organizations by their artificial intelligence capabilities and limits to data sharing. We then discuss why exemplary organizations in different contexts - including public authorities, financial service providers, manufacturing companies, as well as research and development consortia - might consider different approaches to federated learning. To conclude, we argue that federated learning presents organizational challenges with ample interdisciplinary opportunities for information systems researchers.

Computers and Society,Artificial Intelligence,Social and Information Networks
Advances and Open Problems in Federated Learning

Peter Kairouz,H. Brendan McMahan,Brendan Avent,Aurélien Bellet,Mehdi Bennis,Arjun Nitin Bhagoji,Kallista Bonawitz,Zachary Charles,Graham Cormode,Rachel Cummings,Rafael G. L. D’Oliveira,Hubert Eichner,Salim El Rouayheb,David Evans,Josh Gardner,Zachary Garrett,Adrià Gascón,Badih Ghazi,Phillip B. Gibbons,Marco Gruteser,Zaid Harchaoui,Chaoyang He,Lie He,Zhouyuan Huo,Ben Hutchinson,Justin Hsu,Martin Jaggi,Tara Javidi,Gauri Joshi,Mikhail Khodak,Jakub Konecný,Aleksandra Korolova,Farinaz Koushanfar,Sanmi Koyejo,Tancrède Lepoint,Yang Liu,Prateek Mittal,Mehryar Mohri,Richard Nock,Ayfer Özgür,Rasmus Pagh,Hang Qi,Daniel Ramage,Ramesh Raskar,Mariana Raykova,Dawn Song,Weikang Song,Sebastian U. Stich,Ziteng Sun,Ananda Theertha Suresh,Florian Tramèr,Praneeth Vepakomma,Jianyu Wang,Li Xiong,Zheng Xu,Qiang Yang,Felix X. Yu,Han Yu,Sen Zhao,Edited by: Peter Kairouz

DOI: https://doi.org/10.1561/2200000083

2021-01-01

Foundations and Trends® in Machine Learning

Abstract:Federated learning (FL) is a machine learning setting where many clients (e.g. mobile devices orwhole organizations) collaboratively train a model under the orchestration of a central server (e.g. serviceprovider), while keeping the training data decentralized. FL embodies the principles of focused datacollection and minimization, and can mitigate many of the systemic privacy risks and costs resultingfrom traditional, centralized machine learning and data science approaches. Motivated by the explosivegrowth in FL research, this paper discusses recent advances and presents an extensive collection of openproblems and challenges.<h3>Suggested Citation</h3>Edited by: Peter Kairouz and H. Brendan McMahan (2021), "Advances and Open Problems in Federated Learning", Foundations and Trends® in Machine Learning: Vol. 14: No. 1. http://dx.doi.org/10.1561/2200000083

English Else
OpenFed: A Comprehensive and Versatile Open-Source Federated Learning Framework

Dengsheng Chen,Vince Tan,Zhilin Lu,Jie Hu

DOI: https://doi.org/10.48550/arXiv.2109.07852

2023-04-03

Abstract:Recent developments in Artificial Intelligence techniques have enabled their successful application across a spectrum of commercial and industrial settings. However, these techniques require large volumes of data to be aggregated in a centralized manner, forestalling their applicability to scenarios wherein the data is sensitive or the cost of data transmission is prohibitive. Federated Learning alleviates these problems by decentralizing model training, thereby removing the need for data transfer and aggregation. To advance the adoption of Federated Learning, more research and development needs to be conducted to address some important open questions. In this work, we propose OpenFed, an open-source software framework for end-to-end Federated Learning. OpenFed reduces the barrier to entry for both researchers and downstream users of Federated Learning by the targeted removal of existing pain points. For researchers, OpenFed provides a framework wherein new methods can be easily implemented and fairly evaluated against an extensive suite of benchmarks. For downstream users, OpenFed allows Federated Learning to be plugged and play within different subject-matter contexts, removing the need for deep expertise in Federated Learning.

Cryptography and Security,Machine Learning
Recent Advances on Federated Learning: A Systematic Survey

Bingyan Liu,Nuoyan Lv,Yuanchun Guo,Yawen Li

DOI: https://doi.org/10.48550/arXiv.2301.01299

2023-01-03

Abstract:Federated learning has emerged as an effective paradigm to achieve privacy-preserving collaborative learning among different parties. Compared to traditional centralized learning that requires collecting data from each party, in federated learning, only the locally trained models or computed gradients are exchanged, without exposing any data information. As a result, it is able to protect privacy to some extent. In recent years, federated learning has become more and more prevalent and there have been many surveys for summarizing related methods in this hot research topic. However, most of them focus on a specific perspective or lack the latest research progress. In this paper, we provide a systematic survey on federated learning, aiming to review the recent advanced federated methods and applications from different aspects. Specifically, this paper includes four major contributions. First, we present a new taxonomy of federated learning in terms of the pipeline and challenges in federated scenarios. Second, we summarize federated learning methods into several categories and briefly introduce the state-of-the-art methods under these categories. Third, we overview some prevalent federated learning frameworks and introduce their features. Finally, some potential deficiencies of current methods and several future directions are discussed.

Machine Learning
Federated Learning and Wireless Communications

Zhijin Qin,Geoffrey Ye Li,Hao Ye

DOI: https://doi.org/10.1109/mwc.011.2000501

IF: 12.777

2021-10-01

IEEE Wireless Communications

Abstract:Federated learning becomes increasingly attractive in the areas of wireless communications and machine learning due to its powerful learning ability and potential applications. In contrast to other machine learning techniques that require no communication resources, federated learning exploits communications between the central server and the distributed local clients to train and optimize a model. Therefore, how to efficiently assign limited communication resources to train a federated learning model becomes critical to performance optimization. On the other hand, federated learning, as a brand-new tool, can potentially enhance the intelligence of wireless networks. In this article, we provide a comprehensive overview of the relationship between federated learning and wireless communications, including basic principles of federated learning, efficient communications for training a federated learning model, and federated learning for intelligent wireless applications. We also identify some research challenges and directions at the end of this article.

computer science, information systems,telecommunications,engineering, electrical & electronic, hardware & architecture
Federated learning: A cutting-edge survey of the latest advancements and applications

Azim Akhtarshenas,Mohammad Ali Vahedifar,Navid Ayoobi,Behrouz Maham,Tohid Alizadeh,Sina Ebrahimi,David López-Pérez

DOI: https://doi.org/10.1016/j.comcom.2024.107964

IF: 5.047

2024-10-02

Computer Communications

Abstract:Robust machine learning (ML) models can be developed by leveraging large volumes of data and distributing the computational tasks across numerous devices or servers. Federated learning (FL) is a technique in the realm of ML that facilitates this goal by utilizing cloud infrastructure to enable collaborative model training among a network of decentralized devices. Beyond distributing the computational load, FL targets the resolution of privacy issues and the reduction of communication costs simultaneously. To protect user privacy, FL requires users to send model updates rather than transmitting large quantities of raw and potentially confidential data. Specifically, individuals train ML models locally using their own data and then upload the results in the form of weights and gradients to the cloud for aggregation into the global model. This strategy is also advantageous in environments with limited bandwidth or high communication costs, as it prevents the transmission of large data volumes. With the increasing volume of data and rising privacy concerns, alongside the emergence of large-scale ML models like Large Language Models (LLMs), FL presents itself as a timely and relevant solution. It is therefore essential to review current FL algorithms to guide future research that meets the rapidly evolving ML demands. This survey provides a comprehensive analysis and comparison of the most recent FL algorithms, evaluating them on various fronts including mathematical frameworks, privacy protection, resource allocation, and applications. Beyond summarizing existing FL methods, this survey identifies potential gaps, open areas, and future challenges based on the performance reports and algorithms used in recent studies. This survey enables researchers to readily identify existing limitations in the FL field for further exploration.

computer science, information systems,telecommunications,engineering, electrical & electronic
"Federated Learning: Advancements, Applications, and Future Directions for Collaborative Machine Learning in Distributed Environments"

M. Katyayani, Kumar Keshamoni, A. Sree Rama Chandra Murthy, K. Usha Rani, Sreenivasulu Reddy L., Yaswanth Kumar Alapati

DOI: https://doi.org/10.52783/jes.1900

2024-04-13

Abstract:Federated Learning (FL) has become widely recognized as a feasible method for training machine learning models on decentralized devices, ensuring the preservation of data privacy. This study offers an extensive overview of the latest progress in federated learning methods, their applications, and the challenges they entail. We begin by introducing the concept of federated learning and its significance in distributed environments. Next, we delve into a range of methodologies aimed at improving the effectiveness, scalability, and confidentiality of federated learning. These encompass optimization algorithms, communication protocols, and mechanisms designed to uphold privacy. Moreover, we investigate the broad spectrum of applications where federated learning finds utility, spanning healthcare, the Internet of Things (IoT), and edge computing. This exploration illuminates tangible scenarios and advantages in real-world settings. Top of Form Additionally, we analyze the challenges and limitations inherent in federated learning, including communication overhead, non-IID data distribution, and model heterogeneity. We review recent research efforts aimed at addressing these challenges, such as federated averaging variants, adaptive client selection, and robust aggregation techniques. Finally, we outline future research directions and potential avenues for the advancement of federated learning, emphasizing the need for standardized benchmarks, federated learning frameworks, and interdisciplinary collaborations.
Federated Transfer Learning: concept and applications

Sudipan Saha,Tahir Ahmad

DOI: https://doi.org/10.48550/arXiv.2010.15561

2021-03-06

Abstract:Development of Artificial Intelligence (AI) is inherently tied to the development of data. However, in most industries data exists in form of isolated islands, with limited scope of sharing between different organizations. This is an hindrance to the further development of AI. Federated learning has emerged as a possible solution to this problem in the last few years without compromising user privacy. Among different variants of the federated learning, noteworthy is federated transfer learning (FTL) that allows knowledge to be transferred across domains that do not have many overlapping features and users. In this work we provide a comprehensive survey of the existing works on this topic. In more details, we study the background of FTL and its different existing applications. We further analyze FTL from privacy and machine learning perspective.

Machine Learning,Cryptography and Security
A Field Guide to Federated Optimization

Jianyu Wang,Zachary Charles,Zheng Xu,Gauri Joshi,H. Brendan McMahan,Blaise Aguera y Arcas,Maruan Al-Shedivat,Galen Andrew,Salman Avestimehr,Katharine Daly,Deepesh Data,Suhas Diggavi,Hubert Eichner,Advait Gadhikar,Zachary Garrett,Antonious M. Girgis,Filip Hanzely,Andrew Hard,Chaoyang He,Samuel Horvath,Zhouyuan Huo,Alex Ingerman,Martin Jaggi,Tara Javidi,Peter Kairouz,Satyen Kale,Sai Praneeth Karimireddy,Jakub Konecny,Sanmi Koyejo,Tian Li,Luyang Liu,Mehryar Mohri,Hang Qi,Sashank J. Reddi,Peter Richtarik,Karan Singhal,Virginia Smith,Mahdi Soltanolkotabi,Weikang Song,Ananda Theertha Suresh,Sebastian U. Stich,Ameet Talwalkar,Hongyi Wang,Blake Woodworth,Shanshan Wu,Felix X. Yu,Honglin Yuan,Manzil Zaheer,Mi Zhang,Tong Zhang,Chunxiang Zheng,Chen Zhu,Wennan Zhu

DOI: https://doi.org/10.48550/arXiv.2107.06917

2021-07-15

Abstract:Federated learning and analytics are a distributed approach for collaboratively learning models (or statistics) from decentralized data, motivated by and designed for privacy protection. The distributed learning process can be formulated as solving federated optimization problems, which emphasize communication efficiency, data heterogeneity, compatibility with privacy and system requirements, and other constraints that are not primary considerations in other problem settings. This paper provides recommendations and guidelines on formulating, designing, evaluating and analyzing federated optimization algorithms through concrete examples and practical implementation, with a focus on conducting effective simulations to infer real-world performance. The goal of this work is not to survey the current literature, but to inspire researchers and practitioners to design federated learning algorithms that can be used in various practical applications.

Machine Learning

Preface to Federated Learning: Algorithms, Systems, and Applications: Part 2

Introduction to the Special Issue on the Federated Learning: Algorithms, Systems, and Applications: Part 1

Federated Machine Learning: Concept and Applications

A Generalized Look at Federated Learning: Survey and Perspectives

Federated Learning -- Methods, Applications and beyond

Towards Federated Learning at Scale: System Design

Federated Learning: Challenges, Methods, and Future Directions

Federated learning: Overview, strategies, applications, tools and future directions

Federated Learning in Practice: Reflections and Projections

Preconditioned Federated Learning

Federated Select: A Primitive for Communication- and Memory-Efficient Federated Learning

Federated Learning: Organizational Opportunities, Challenges, and Adoption Strategies

Advances and Open Problems in Federated Learning

OpenFed: A Comprehensive and Versatile Open-Source Federated Learning Framework

Recent Advances on Federated Learning: A Systematic Survey

Federated Learning and Wireless Communications

Federated learning: A cutting-edge survey of the latest advancements and applications

"Federated Learning: Advancements, Applications, and Future Directions for Collaborative Machine Learning in Distributed Environments"

Federated Transfer Learning: concept and applications

A Field Guide to Federated Optimization