Trustworthy AI Using Confidential Federated Learning

Jinnan Guo,Peter Pietzuch,Andrew Paverd,Kapil Vaswani

DOI: https://doi.org/10.1145/3677390

IF: 22.7

2024-08-27

Communications of the ACM

Abstract:Classic federated learning emphasizes security and privacy, at the cost of transparency and accountability. Confidential Federated Learning addresses this gap.

computer science, theory & methods, software engineering, hardware & architecture

What problem does this paper attempt to address?

A Three-layer Security Assurance Model for a Decentralized Federated Learning System

Jinyu Liu,Ziyang Wang,Qing Yang,Sissi Xiaoxiao Wu

DOI: https://doi.org/10.1109/wocc55104.2022.9880591

2022-01-01

Abstract:We focus on the security issues of decentralized federated learning systems for the core requirements of secure environments and trusted computing for applications such as medical big data, distributed cognitive learning, and autonomous driving. We propose a three-layer security assurance model that aims to ensure secure trusted computing in decentralized networks. Therein, the first-layer security model realizes the authentication of trusted users through blockchain technology to ensure the safe access of legitimate users to the greatest extent; the second-layer security model utilizes the trusted execution environment (TEE) technology to realize the privacy protection and trusted computing of each local user; the last layer of security model adopts a set of detection and location algorithms for defending the internal attackers, in case the first two layers of models fail. Our experimental results show that the proposed three-layer security model can effectively defend against external and internal attacks in the network, thereby promoting secure and trusted computing in distributed federated learning networks.
Trustworthy machine learning in the context of security and privacy

Ramesh Upreti,Pedro G. Lind,Ahmed Elmokashfi,Anis Yazidi

DOI: https://doi.org/10.1007/s10207-024-00813-3

2024-04-03

International Journal of Information Security

Abstract:Abstract Artificial intelligence-based algorithms are widely adopted in critical applications such as healthcare and autonomous vehicles. Mitigating the security and privacy issues of AI models, and enhancing their trustworthiness have become of paramount importance. We present a detailed investigation of existing security, privacy, and defense techniques and strategies to make machine learning more secure and trustworthy. We focus on the new paradigm of machine learning called federated learning, where one aims to develop machine learning models involving different partners (data sources) that do not need to share data and information with each other. In particular, we discuss how federated learning bridges security and privacy, how it guarantees privacy requirements of AI applications, and then highlight challenges that need to be addressed in the future. Finally, after having surveyed the high-level concepts of trustworthy AI and its different components and identifying present research trends addressing security, privacy, and trustworthiness separately, we discuss possible interconnections and dependencies between these three fields. All in all, we provide some insight to explain how AI researchers should focus on building a unified solution combining security, privacy, and trustworthy AI in the future.

computer science, information systems, theory & methods, software engineering
A Survey of Trustworthy Federated Learning: Issues, Solutions, and Challenges

Yifei Zhang,Dun Zeng,Jinglong Luo,Xinyu Fu,Guanzhong Chen,Zenglin Xu,Irwin King

DOI: https://doi.org/10.1145/3678181

IF: 5

2024-07-23

ACM Transactions on Intelligent Systems and Technology

Abstract:Trustworthy Artificial Intelligence (TAI) has proven invaluable in curbing potential negative repercussions tied to AI applications. Within the TAI spectrum, Federated Learning (FL) emerges as a promising solution to safeguard personal information in distributed settings across a multitude of practical contexts. However, the realm of FL is not without its challenges. Especially worrisome are adversarial attacks targeting its algorithmic robustness and systemic confidentiality. Moreover, the presence of biases and opacity in prediction outcomes further complicates FL’s broader adoption. Consequently, there is a growing expectation for FL to instill trust. To address this, we chart out a comprehensive road-map for Trustworthy Federated Learning (TFL) and provide an overview of existing efforts across four pivotal dimensions: Privacy & Security , Robustness , Fairness , and Explainability . For each dimension, we identify potential pitfalls that might undermine TFL and present a curated selection of defensive strategies, enriched by a discourse on technical solutions tailored for TFL. Furthermore, we present potential challenges and future directions to be explored for in-depth TFL research with broader impacts.

computer science, information systems, artificial intelligence
A Blockchain-Based Trustworthy Federated Learning Privacy Protection Scheme

Jianan Su,Ai Gu,Xiaowei Liu,Xiaohui Li,Siting Lv

DOI: https://doi.org/10.1109/WCNC57260.2024.10570990

2024-04-21

Abstract:With the emergence of distributed machine learning, federated learning has become a prevalent paradigm for multi-party collaborative training scenarios. To address security vulnerabilities such as imperfect encryption and incapability to with-stand attacks within existing federated learning architectures, a blockchain-based trustworthy federated learning architecture is proposed. The architecture utilizes a threshold variant of the Paillier encryption scheme to protect the local and global model updates. A flexible Multi-Krum selection scheme is also proposed for balancing the utility of the overall architecture while ensuring safe federated learning training processes. Based on theoretical analysis and experimental results, the proposed architecture demonstrates a high level of security and the feasibility of the scheme is supported.

Engineering,Computer Science
DeTrust-FL: Privacy-Preserving Federated Learning in Decentralized Trust Setting

Runhua Xu,Nathalie Baracaldo,Yi Zhou,Ali Anwar,Swanand Kadhe,Heiko Ludwig

DOI: https://doi.org/10.48550/arXiv.2207.07779

2022-07-16

Abstract:Federated learning has emerged as a privacy-preserving machine learning approach where multiple parties can train a single model without sharing their raw training data. Federated learning typically requires the utilization of multi-party computation techniques to provide strong privacy guarantees by ensuring that an untrusted or curious aggregator cannot obtain isolated replies from parties involved in the training process, thereby preventing potential inference attacks. Until recently, it was thought that some of these secure aggregation techniques were sufficient to fully protect against inference attacks coming from a curious aggregator. However, recent research has demonstrated that a curious aggregator can successfully launch a disaggregation attack to learn information about model updates of a target party. This paper presents DeTrust-FL, an efficient privacy-preserving federated learning framework for addressing the lack of transparency that enables isolation attacks, such as disaggregation attacks, during secure aggregation by assuring that parties' model updates are included in the aggregated model in a private and secure manner. DeTrust-FL proposes a decentralized trust consensus mechanism and incorporates a recently proposed decentralized functional encryption (FE) scheme in which all parties agree on a participation matrix before collaboratively generating decryption key fragments, thereby gaining control and trust over the secure aggregation process in a decentralized setting. Our experimental evaluation demonstrates that DeTrust-FL outperforms state-of-the-art FE-based secure multi-party aggregation solutions in terms of training time and reduces the volume of data transferred. In contrast to existing approaches, this is achieved without creating any trust dependency on external trusted entities.

Cryptography and Security
Blockchain-based Trustworthy Federated Learning Architecture

Sin Kit Lo,Yue Liu,Qinghua Lu,Chen Wang,Xiwei Xu,Hye-Young Paik,Liming Zhu

DOI: https://doi.org/10.48550/arXiv.2108.06912

2021-10-29

Abstract:Federated learning is an emerging privacy-preserving AI technique where clients (i.e., organisations or devices) train models locally and formulate a global model based on the local model updates without transferring local data externally. However, federated learning systems struggle to achieve trustworthiness and embody responsible AI principles. In particular, federated learning systems face accountability and fairness challenges due to multi-stakeholder involvement and heterogeneity in client data distribution. To enhance the accountability and fairness of federated learning systems, we present a blockchain-based trustworthy federated learning architecture. We first design a smart contract-based data-model provenance registry to enable accountability. Additionally, we propose a weighted fair data sampler algorithm to enhance fairness in training data. We evaluate the proposed approach using a COVID-19 X-ray detection use case. The evaluation results show that the approach is feasible to enable accountability and improve fairness. The proposed algorithm can achieve better performance than the default federated learning setting in terms of the model's generalisation and accuracy.

Machine Learning,Artificial Intelligence
CFSL: A Credible Federated Self-Learning Framework

Weishan Zhang,Zhicheng Bao,Yuru Liu,Liang Xu,Qinghua Lu,Huansheng Ning,Xiao Wang,Su Yang,Fei-Yue Wang,Zengxiang Li

DOI: https://doi.org/10.1109/jiot.2023.3286398

IF: 10.6

2023-01-01

IEEE Internet of Things Journal

Abstract:Federated learning can collaboratively train AI models while protecting data privacy. In practical industry environment, Non-IID characteristics of data affect the effectiveness of federated learning. Personalized federated learning can help resolve this, but it cannot adapt to unknown data. In addition, practical applications also call for trusted training environment and remain stable when there are security threats. In this paper, we propose a Credible Federated Self-Learning (CFSL), based on the idea of hypernetwork supported by blockchain to achieve secured, credible, personalized federated self-learning, especially, for unknown data in Non-IID environment. Extensive experiments on three Non-IID datasets demonstrate the capabilities on adaptive resilience for security attacks and on accuracy of recognizing unknown objects, with good performance at the same time. CFSL outperforms the existing personalized federated learning methods, with an increase in average accuracy by 4.11%.

computer science, information systems,telecommunications,engineering, electrical & electronic
Federated Learning for Privacy-Preserving AI

Yong Cheng,Yang Liu,Tianjian Chen,Qiang Yang

DOI: https://doi.org/10.1145/3387107

IF: 22.7

2020-01-01

Communications of the ACM

Abstract:Engineering and algorithmic framework to ensure data privacy and user confidentiality.
Secure and Efficient Decentralized Federated Learning with Data Representation Protection

Z. H. Qin,Shuiguang Deng,Xin Yan,Schahram Dustdar,Albert Y. Zomaya

DOI: https://doi.org/10.48550/arxiv.2205.10568

2022-01-01

Abstract:Federated learning (FL) is a promising technical support to the vision of ubiquitous artificial intelligence in the sixth generation (6G) wireless communication network. However, traditional FL heavily relies on a trusted centralized server. Besides, FL is vulnerable to poisoning attacks, and the global aggregation of model updates makes the private training data under the risk of being reconstructed. What's more, FL suffers from efficiency problem due to heavy communication cost. Although decentralized FL eliminates the problem of the central dependence of traditional FL, it makes other problems more serious. In this paper, we propose BlockDFL, an efficient fully peer-to-peer (P2P) framework for decentralized FL. It integrates gradient compression and our designed voting mechanism with blockchain to efficiently coordinate multiple peer participants without mutual trust to carry out decentralized FL, while preventing data from being reconstructed according to transmitted model updates. Extensive experiments conducted on two real-world datasets exhibit that BlockDFL obtains competitive accuracy compared to centralized FL and can defend against poisoning attacks while achieving efficiency and scalability. Especially when the proportion of malicious participants is as high as 40 percent, BlockDFL can still preserve the accuracy of FL, which outperforms existing fully decentralized FL frameworks.
Confidential Federated Computations

Hubert Eichner,Daniel Ramage,Kallista Bonawitz,Dzmitry Huba,Tiziano Santoro,Brett McLarnon,Timon Van Overveldt,Nova Fallen,Peter Kairouz,Albert Cheu,Katharine Daly,Adria Gascon,Marco Gruteser,Brendan McMahan

2024-04-17

Abstract:Federated Learning and Analytics (FLA) have seen widespread adoption by technology platforms for processing sensitive on-device data. However, basic FLA systems have privacy limitations: they do not necessarily require anonymization mechanisms like differential privacy (DP), and provide limited protections against a potentially malicious service provider. Adding DP to a basic FLA system currently requires either adding excessive noise to each device's updates, or assuming an honest service provider that correctly implements the mechanism and only uses the privatized outputs. Secure multiparty computation (SMPC) -based oblivious aggregations can limit the service provider's access to individual user updates and improve DP tradeoffs, but the tradeoffs are still suboptimal, and they suffer from scalability challenges and susceptibility to Sybil attacks. This paper introduces a novel system architecture that leverages trusted execution environments (TEEs) and open-sourcing to both ensure confidentiality of server-side computations and provide externally verifiable privacy properties, bolstering the robustness and trustworthiness of private federated computations.

Cryptography and Security,Machine Learning
Privacy and Trust Redefined in Federated Machine Learning

Pavlos Papadopoulos,Will Abramson,Adam J. Hall,Nikolaos Pitropakis,William J. Buchanan

DOI: https://doi.org/10.3390/make3020017

2021-03-30

Abstract:A common privacy issue in traditional machine learning is that data needs to be disclosed for the training procedures. In situations with highly sensitive data such as healthcare records, accessing this information is challenging and often prohibited. Luckily, privacy-preserving technologies have been developed to overcome this hurdle by distributing the computation of the training and ensuring the data privacy to their owners. The distribution of the computation to multiple participating entities introduces new privacy complications and risks. In this paper, we present a privacy-preserving decentralised workflow that facilitates trusted federated learning among participants. Our proof-of-concept defines a trust framework instantiated using decentralised identity technologies being developed under Hyperledger projects Aries/Indy/Ursa. Only entities in possession of Verifiable Credentials issued from the appropriate authorities are able to establish secure, authenticated communication channels authorised to participate in a federated learning workflow related to mental health data.

Cryptography and Security,Computers and Society,Distributed, Parallel, and Cluster Computing,Machine Learning
A Secure Federated Transfer Learning Framework

Yang Liu,Yan Kang,Chaoping Xing,Tianjian Chen,Qiang Yang

DOI: https://doi.org/10.1109/mis.2020.2988525

IF: 6.744

2020-01-01

IEEE Intelligent Systems

Abstract:Machine learning relies on the availability of vast amounts of data for training. However, in reality, data are mostly scattered across different organizations and cannot be easily integrated due to many legal and practical constraints. To address this important challenge in the field of machine learning, we introduce a new technique and framework, known as federated transfer learning (FTL), to improve statistical modeling under a data federation. FTL allows knowledge to be shared without compromising user privacy and enables complementary knowledge to be transferred across domains in a data federation, thereby enabling a target-domain party to build flexible and effective models by leveraging rich labels from a source domain. This framework requires minimal modifications to the existing model structure and provides the same level of accuracy as the nonprivacy-preserving transfer learning. It is flexible and can be effectively adapted to various secure multiparty machine learning tasks.
From distributed machine learning to federated learning: In the view of data privacy and security

Sheng Shen,Tianqing Zhu,Di Wu,Wei Wang,Wanlei Zhou

DOI: https://doi.org/10.1002/cpe.6002

2020-09-23

Concurrency and Computation: Practice and Experience

Abstract:<p>Federated learning is an improved version of distributed machine learning that further offloads operations which would usually be performed by a central server. The server becomes more like an assistant coordinating clients to work together rather than micromanaging the workforce as in traditional DML. One of the greatest advantages of federated learning is the additional privacy and security guarantees it affords. Federated learning architecture relies on smart devices, such as smartphones and IoT sensors, that collect and process their own data, so sensitive information never has to leave the client device. Rather, clients train a submodel locally and send an encrypted update to the central server for aggregation into the global model. These strong privacy guarantees make federated learning an attractive choice in a world where data breaches and information theft are common and serious threats. This survey outlines the landscape and latest developments in data privacy and security for federated learning. We identify the different mechanisms used to provide privacy and security, such as differential privacy, secure multiparty computation and secure aggregation. We also survey the current attack models, identifying the areas of vulnerability and the strategies adversaries use to penetrate federated systems. The survey concludes with a discussion on the open challenges and potential directions of future work in this increasingly popular learning paradigm.</p>

English Else
Federated Learning with Privacy-preserving and Model IP-right-protection

Qiang Yang,Anbu Huang,Lixin Fan,Chee Seng Chan,Jian Han Lim,Kam Woh Ng,Ding Sheng Ong,Bowen Li

DOI: https://doi.org/10.1007/s11633-022-1343-2

2023-01-11

Machine Intelligence Research

Abstract:In the past decades, artificial intelligence (AI) has achieved unprecedented success, where statistical models become the central entity in AI. However, the centralized training and inference paradigm for building and using these models is facing more and more privacy and legal challenges. To bridge the gap between data privacy and the need for data fusion, an emerging AI paradigm federated learning (FL) has emerged as an approach for solving data silos and data privacy problems. Based on secure distributed AI, federated learning emphasizes data security throughout the lifecycle, which includes the following steps: data preprocessing, training, evaluation, and deployments. FL keeps data security by using methods, such as secure multi-party computation (MPC), differential privacy, and hardware solutions, to build and use distributed multiple-party machine-learning systems and statistical models over different data sources. Besides data privacy concerns, we argue that the concept of "model" matters, when developing and deploying federated models, they are easy to expose to various kinds of risks including plagiarism, illegal copy, and misuse. To address these issues, we introduce FedIPR, a novel ownership verification scheme, by embedding watermarks into FL models to verify the ownership of FL models and protect model intellectual property rights (IPR or IP-right for short). While security is at the core of FL, there are still many articles referred to distributed machine learning with no security guarantee as "federated learning", which are not satisfied with the FL definition supposed to be. To this end, in this paper, we reiterate the concept of federated learning and propose secure federated learning (SFL), where the ultimate goal is to build trustworthy and safe AI with strong privacy-preserving and IP-right-preserving. We provide a comprehensive overview of existing works, including threats, attacks, and defenses in each phase of SFL from the lifecycle perspective.
A Secure and Fair Federated Learning Framework Based on Consensus Incentive Mechanism

Feng Zhu,Feng Hu,Yanchao Zhao,Bing Chen,Xiaoyang Tan

DOI: https://doi.org/10.3390/math12193068

IF: 2.4

2024-09-30

Mathematics

Abstract:Federated learning facilitates collaborative computation among multiple participants while safeguarding user privacy. However, current federated learning algorithms operate under the assumption that all participants are trustworthy and their systems are secure. Nonetheless, real-world scenarios present several challenges: (1) Malicious clients disrupt federated learning through model poisoning and data poisoning attacks. Although some research has proposed secure aggregation methods to address this issue, many methods have inherent limitations. (2) Clients may refuse or passively participate in the training process due to considerations of self-interest, and may even interfere with the training process due to competitive relationships. To overcome these obstacles, we have devised a reliable federated framework aimed at ensuring secure computing throughout the entirety of federated task processes. Initially, we propose a method for detecting malicious models to safeguard the integrity of model aggregation. Furthermore, we have proposed a fair contribution assessment method and awarded the right to write blocks to the creator of the optimal model, ensuring the active participation of participants in both local training and model aggregation. Finally, we establish a computational framework grounded in blockchain and smart contracts to uphold the integrity and fairness of federated tasks. To assess the efficacy of our framework, we conduct simulations involving various types of client attacks and contribution assessment scenarios using multiple open-source datasets. Results from these experiments demonstrate that our framework effectively ensures the credibility of federated tasks while achieving impartial evaluation of client contributions.

mathematics
EPFed: Achieving Optimal Balance between Privacy and Efficiency in Federated Learning

Dong Mao,Qiongqian Yang,Hongkai Wang,Zuge Chen,Chen Li,Yubo Song,Zhongyuan Qin

DOI: https://doi.org/10.3390/electronics13061028

IF: 2.9

2024-03-10

Electronics

Abstract:Federated learning (FL) is increasingly challenged by security and privacy concerns, particularly vulnerabilities exposed by malicious participants. There remains a gap in effectively countering threats such as model inversion and poisoning attacks in existing research. To address these challenges, this paper proposes the Effective Private-Protected Federated Learning Aggregation Algorithm (EPFed), a framework that utilizes a blockchain platform, homomorphic encryption, and secret sharing to fortify the data privacy and computational efficiency in a federated learning environment. EPFed works by establishing "trust groups" through the unique integration of a Chinese Remainder Theorem-based secret sharing scheme with Paillier homomorphic encryption, streamlining secure model parameter exchange and aggregation while minimizing the computational load. Our performance-driven aggregation strategy leverages local performance metrics to safeguard against malicious contributions, ensuring both the integrity and efficiency of the learning process. The evaluations demonstrate that EPFed achieves a remarkable accuracy rate of 92.5%, thereby confirming the advanced nature of the proposed solution in addressing the pressing challenges of FL.

engineering, electrical & electronic,computer science, information systems,physics, applied
Decentralized Federated Learning: A Survey on Security and Privacy

Ehsan Hallaji,Roozbeh Razavi-Far,Mehrdad Saif,Boyu Wang,Qiang Yang

DOI: https://doi.org/10.1109/tbdata.2024.3362191

2024-01-01

IEEE Transactions on Big Data

Abstract:Federated learning has been rapidly evolving and gaining popularity in recent years due to its privacy-preserving features, among other advantages. Nevertheless, the exchange of model updates and gradients in this architecture provides new attack surfaces for malicious users of the network which may jeopardize the model performance and user and data privacy. For this reason, one of the main motivations for decentralized federated learning is to eliminate server-related threats by removing the server from the network and compensating for it through technologies such as blockchain. However, this advantage comes at the cost of challenging the system with new privacy threats. Thus, performing a thorough security analysis in this new paradigm is necessary. This survey studies possible variations of threats and adversaries in decentralized federated learning and overviews the potential defense mechanisms. Trustability and verifiability of decentralized federated learning are also considered in this study.

computer science, information systems, theory & methods
Efficient and Privacy-Enhanced Federated Learning for Industrial Artificial Intelligence

Meng Hao,Hongwei Li,Xizhao Luo,Guowen Xu,Haomiao Yang,Sen Liu

DOI: https://doi.org/10.1109/tii.2019.2945367

IF: 12.3

2020-10-01

IEEE Transactions on Industrial Informatics

Abstract:By leveraging deep learning-based technologies, industrial artificial intelligence (IAI) has been applied to solve various industrial challenging problems in Industry 4.0. However, for privacy reasons, traditional centralized training may be unsuitable for sensitive data-driven industrial scenarios, such as healthcare and autopilot. Recently, federated learning has received widespread attention, since it enables participants to collaboratively learn a shared model without revealing their local data. However, studies have shown that, by exploiting the shared parameters adversaries can still compromise industrial applications such as auto-driving navigation systems, medical data in wearable devices, and industrial robots' decision making. In this article, to solve this problem, we propose an efficient and privacy-enhanced federated learning (PEFL) scheme for IAI. Compared with existing solutions, PEFL is noninteractive, and can prevent private data from being leaked even if multiple entities collude with each other. Moreover, extensive experiments with real-world data demonstrate the superiority of PEFL in terms of accuracy and efficiency.

automation & control systems,computer science, interdisciplinary applications,engineering, industrial

Trustworthy AI Using Confidential Federated Learning

A Three-layer Security Assurance Model for a Decentralized Federated Learning System

Trustworthy machine learning in the context of security and privacy

A Survey of Trustworthy Federated Learning: Issues, Solutions, and Challenges

A Blockchain-Based Trustworthy Federated Learning Privacy Protection Scheme

DeTrust-FL: Privacy-Preserving Federated Learning in Decentralized Trust Setting

Blockchain-based Trustworthy Federated Learning Architecture

CFSL: A Credible Federated Self-Learning Framework

Federated Learning for Privacy-Preserving AI

Secure and Efficient Decentralized Federated Learning with Data Representation Protection

Confidential Federated Computations

Privacy and Trust Redefined in Federated Machine Learning

A Secure Federated Transfer Learning Framework

From distributed machine learning to federated learning: In the view of data privacy and security

Federated Learning with Privacy-preserving and Model IP-right-protection

A Secure and Fair Federated Learning Framework Based on Consensus Incentive Mechanism

EPFed: Achieving Optimal Balance between Privacy and Efficiency in Federated Learning

Decentralized Federated Learning: A Survey on Security and Privacy

Efficient and Privacy-Enhanced Federated Learning for Industrial Artificial Intelligence