Abstract:Federated Learning (FL) has gained attention for addressing data scarcity and privacy concerns. While parallel FL algorithms like FedAvg exhibit remarkable performance, they face challenges in scenarios with diverse network speeds and concerns about centralized control, especially in multi-institutional collaborations like the medical domain. Serial FL presents an alternative solution, circumventing these challenges by transferring model updates serially between devices in a cyclical manner. Nevertheless, it is deemed inferior to parallel FL in that (1) its performance shows undesirable fluctuations, and (2) it converges to a lower plateau, particularly when dealing with non-IID data. The observed phenomenon is attributed to catastrophic forgetting due to knowledge loss from previous sites. In this paper, to overcome fluctuation and low efficiency in the iterative learning and forgetting process, we introduce cyclical weight consolidation (CWC), a straightforward yet potent approach specifically tailored for serial FL. CWC employs a consolidation matrix to regulate local optimization. This matrix tracks the significance of each parameter on the overall federation throughout the entire training trajectory, preventing abrupt changes in significant weights. During revisitation, to maintain adaptability, old memory undergoes decay to incorporate new information. Our comprehensive evaluations demonstrate that in various non-IID settings, CWC mitigates the fluctuation behavior of the original serial FL approach and enhances the converged performance consistently and significantly. The improved performance is either comparable to or better than the parallel vanilla.

Addressing Catastrophic Forgetting in Federated Learning on Resource-Constrained Devices: A Feature Replay Approach

Towards Efficient Replay in Federated Incremental Learning

Diffusion-Driven Data Replay: A Novel Approach to Combat Forgetting in Federated Class Continual Learning

Cyclical Weight Consolidation: Towards Solving Catastrophic Forgetting in Serial Federated Learning

TARGET: Federated Class-Continual Learning Via Exemplar-Free Distillation

Don't Memorize; Mimic The Past: Federated Class Incremental Learning Without Episodic Memory

FedFRR: Federated Forgetting-Resistant Representation Learning

Overcoming Forgetting in Federated Learning on Non-IID Data

Resource-Efficient Heterogenous Federated Continual Learning on Edge

RRA-FFSCIL: Inter-intra classes representation and relationship augmentation federated few-shot incremental learning

Overcoming Spatial-Temporal Catastrophic Forgetting for Federated Class-Incremental Learning

Knowledge Lock: Overcoming Catastrophic Forgetting in Federated Learning

Overcoming Catastrophic Forgetting in Federated Class-Incremental Learning via Federated Global Twin Generator

Efficient Knowledge Management for Heterogeneous Federated Continual Learning on Resource-Constrained Edge Devices

Efficient knowledge management for heterogenous federated continual learning on resource-constrained edge devices

SR-FDIL: Synergistic Replay for Federated Domain-Incremental Learning

Federated Learning and catastrophic forgetting in pervasive computing: demonstration in HAR domain

CORE: Mitigating Catastrophic Forgetting in Continual Learning through Cognitive Replay

Learn to Grow: A Continual Structure Learning Framework for Overcoming Catastrophic Forgetting

Accurate Forgetting for Heterogeneous Federated Continual Learning

Rehearsal-free Federated Domain-incremental Learning