Federated Continual Learning for Edge-AI: A Comprehensive Survey

Zi Wang,Fei Wu,Feng Yu,Yurui Zhou,Jia Hu,Geyong Min
2024-11-21
Abstract:Edge-AI, the convergence of edge computing and artificial intelligence (AI), has become a promising paradigm that enables the deployment of advanced AI models at the network edge, close to users. In Edge-AI, federated continual learning (FCL) has emerged as an imperative framework, which fuses knowledge from different clients while preserving data privacy and retaining knowledge from previous tasks as it learns new ones. By so doing, FCL aims to ensure stable and reliable performance of learning models in dynamic and distributed environments. In this survey, we thoroughly review the state-of-the-art research and present the first comprehensive survey of FCL for Edge-AI. We categorize FCL methods based on three task characteristics: federated class continual learning, federated domain continual learning, and federated task continual learning. For each category, an in-depth investigation and review of the representative methods are provided, covering background, challenges, problem formalisation, solutions, and limitations. Besides, existing real-world applications empowered by FCL are reviewed, indicating the current progress and potential of FCL in diverse application domains. Furthermore, we discuss and highlight several prospective research directions of FCL such as algorithm-hardware co-design for FCL and FCL with foundation models, which could provide insights into the future development and practical deployment of FCL in the era of Edge-AI.
Machine Learning,Artificial Intelligence,Distributed, Parallel, and Cluster Computing,Networking and Internet Architecture
What problem does this paper attempt to address?
### What problems does this paper attempt to solve? This review paper, "Federated Continual Learning for Edge - AI: A Comprehensive Survey", aims to explore and summarize the latest research progress of Federated Continual Learning (FCL) in Edge - AI. Specifically, the paper attempts to solve the following key problems: 1. **Privacy Protection and Data Distribution Problems**: - **Privacy Protection**: Traditional deep - learning model training relies on centralized cloud servers, which brings the risk of user data privacy leakage. FCL reduces the dependence on central servers by bringing computing and storage resources closer to the data source (i.e., edge devices), thereby enhancing data privacy protection. - **Uneven Data Distribution**: In practical applications, the data distribution of different clients is often non - independent and identically distributed (non - IID), which poses challenges to model training and aggregation. FCL enables each client to train locally through a distributed learning framework and share knowledge through federated learning. 2. **Continual Learning and Catastrophic Forgetting**: - **Continual Learning**: Data and tasks in the real world are dynamically changing, and the model needs to keep learning new tasks while retaining the knowledge of old tasks. FCL combines the advantages of federated learning and continual learning, enabling the model to adapt to new tasks and data streams in a distributed environment. - **Catastrophic Forgetting**: When the model learns new tasks, it may forget the previously learned knowledge. An important goal of FCL is to reduce this catastrophic forgetting and ensure that the model maintains good performance on old tasks while learning new tasks. 3. **Application Scenarios and Future Directions**: - **Application Scenarios**: The paper also reviews the practical applications of FCL in intelligent transportation systems, intelligent medical systems, the Internet of Things (IoT), unmanned aerial vehicles (UAVs), intelligent energy, digital twins, etc., demonstrating its wide applicability and potential. - **Future Research Directions**: The authors discuss several open - ended challenges faced by FCL, such as the lack of a general benchmark, interpretability, algorithm - hardware co - design, and combination with base models, and propose future research directions to promote the rapid development and wide application of FCL in the Edge - AI era. ### Summary Through a comprehensive survey of FCL, this paper not only summarizes the achievements and deficiencies of existing research but also provides valuable guidance for future research. It pays special attention to how to achieve stable and reliable model performance in a dynamic and distributed Edge - AI environment while ensuring data privacy and the ability to learn continuously.