Abstract:Fog computing emerged as a promising paradigm to address the challenges of processing and managing data generated by the Internet of Things (IoT). Load balancing (LB) plays a crucial role in Fog computing environments to optimize the overall system performance. It requires efficient resource allocation to improve resource utilization, minimize latency, and enhance the quality of service for end-users. In this work, we improve the performance of privacy-aware Reinforcement Learning (RL) agents that optimize the execution delay of IoT applications by minimizing the waiting delay. To maintain privacy, these agents optimize the waiting delay by minimizing the change in the number of queued requests in the whole system, i.e., without explicitly observing the actual number of requests that are queued in each Fog node nor observing the compute resource capabilities of those nodes. Besides improving the performance of these agents, we propose in this paper a lifelong learning framework for these agents, where lightweight inference models are used during deployment to minimize action delay and only retrained in case of significant environmental changes. To improve the performance, minimize the training cost, and adapt the agents to those changes, we explore the application of Transfer Learning (TL). TL transfers the knowledge acquired from a source domain and applies it to a target domain, enabling the reuse of learned policies and experiences. TL can be also used to pre-train the agent in simulation before fine-tuning it in the real environment; this significantly reduces failure probability compared to learning from scratch in the real environment. To our knowledge, there are no existing efforts in the literature that use TL to address lifelong learning for RL-based Fog LB; this is one of the main obstacles in deploying RL LB solutions in Fog systems.

Enhancing Learning Efficiency in FACL: A Novel Fuzzy Rule Transfer Method for Transfer Learning

Transferring knowledge from human-demonstration trajectories to reinforcement learning

Efficient Deep Reinforcement Learning Via Adaptive Policy Transfer

Efficient Deep Reinforcement Learning Through Policy Transfer.

Shaping in Reinforcement Learning Via Knowledge Transferred from Human-Demonstrations

Federated Transfer Reinforcement Learning for Autonomous Driving

CLFR-M: Continual Learning Framework for Robots Via Human Feedback and Dynamic Memory

Heuristically Accelerated Reinforcement Learning by Means of Case-Based Reasoning and Transfer Learning

Expert-Free Online Transfer Learning in Multi-Agent Reinforcement Learning

Fuzzy Feedback Multi-Agent Reinforcement Learning for Adversarial Dynamic Multi-Team Competitions

Decision-Focused Model-based Reinforcement Learning for Reward Transfer

Fuzzy advantage learning

Skill based transfer learning with domain adaptation for continuous reinforcement learning domains

Enhancing team performance with transfer-learning during real-world human-robot collaboration

Lifelong Learning for Fog Load Balancing: A Transfer Learning Approach

Transfer of Temporal Logic Formulas in Reinforcement Learning

Task Transfer by Preference-Based Cost Learning.

Foundations for Transfer in Reinforcement Learning: A Taxonomy of Knowledge Modalities

CAT: Caution Aware Transfer in Reinforcement Learning via Distributional Risk

Learning To Walk With Prior Knowledge

Efficient Exploration for Multi-Agent Reinforcement Learning Via Transferable Successor Features