Abstract:In the field of Natural Language Processing (NLP), Deep Learning (DL) and Neural Network (NN) technologies have been widely applied to machine translation and sentiment analysis and have demonstrated outstanding performance. In recent years, NLP applications have also combined multimodal data, such as visual and audio, continuously improving language processing performance. At the same time, the size of Neural Network models is increasing, and many models cannot be deployed on devices with limited computing resources. Deploying models on cloud platforms has become a trend. However, deploying models in the cloud introduces new privacy risks for endpoint data, despite overcoming computational limitations. Federated Learning (FL) methods protect local data by keeping the data on the client side and only sending local updates to the central server. However, the FL architecture still has problems, such as vulnerability to adversarial attacks and non-IID data distribution. In this work, we propose a Federated Learning aggregation method called FedREAS. The server uses a benchmark dataset to train a global model and obtains benchmark updates in this method. Before aggregating local updates, the server adjusts the local updates using the benchmark updates and then returns the adjusted benchmark updates. Then, based on the similarity between the adjusted local updates and the adjusted benchmark updates, the server aggregates these local updates to obtain a more robust update. This method also improves the client selection process. FedREAS selects suitable clients for training at the beginning of each round based on specific strategies, the similarity of the previous round’s updates, and the submitted data. We conduct experiments on different datasets and compare FedREAS with other Federated Learning methods. The results show that FedREAS outperforms other methods regarding model performance and resistance to attacks.

Federated Adaptation of Reservoirs via Intrinsic Plasticity

Regular Echo State Networks

FedEmb: A Vertical and Hybrid Federated Learning Algorithm using Network And Feature Embedding Aggregation

Federated Online Adaptation for Deep Stereo

Learning the Deterministically Constructed Echo State Networks.

AsyncFedED: Asynchronous Federated Learning with Euclidean Distance Based Adaptive Weight Aggregation

Exploring deep echo state networks for image classification: a multi-reservoir approach

An EMD-Based Adaptive Client Selection Algorithm for Federated Learning in Heterogeneous Data Scenarios

Communication-Efficient Learning of Deep Networks from Decentralized Data

An on-line training algorithm for echo state networks with extended kalman filter

FedECADO: A Dynamical System Model of Federated Learning

Leveraging Function Space Aggregation for Federated Learning at Scale

Modelling and evaluating restricted ESNs on single- and multi-timescale problems

Echo state networks regulated by local intrinsic plasticity rules for regression

Ensemble and Pre-Training Approach for Echo State Network and Extreme Learning Machine Models

A Decentralized Training Algorithm for Echo State Networks in Distributed Big Data Applications

Federated Learning Under Intermittent Client Availability and Time-Varying Communication Constraints

FedREAS: A Robust Efficient Aggregation and Selection Framework for Federated Learning

One-shot Federated Learning Without Server-side Training

FedSW: Federated learning with adaptive sample weights

Multi-Reservoir Echo State Network with Five-Elements Cycle