Abstract:It is difficult for resource-constrained edge servers to simultaneously meet the performance requirements of all the latency-sensitive Internet of Things (IoT) applications in edge computing. Therefore, it is a significant challenge to efficiently generate a task offloading strategy. Recently, deep reinforcement learning (DRL)-based task offloading methods have been studied to ensure long-term performance optimization. However, there are challenges in existing DRL-based task offloading methods, such as insufficient sample diversity and high exploration cost. To optimize the performance of edge computing and facilitate the development and deployment of event-driven IoT applications, the serverless edge computing model has emerged. It combines serverless computing, also known as Function as a Service (FaaS), with edge computing and has been adopted in edge AI inference and prediction, stream processing, face recognition, etc. In this paper, an experience-sharing deep reinforcement learning-based distributed function offloading method called ES-DRL is proposed in the setting of a combined stateful and stateless execution model for serverless edge computing. ES-DRL adopts a distributed learning architecture, where each edge FaaS (EFaaS) obtains the current state of the local environment and inputs them to the local DRL agent, which outputs the function offloading strategy. Then, each EFaaS uploads the experience data interacting with the environment to a global shared replay buffer located in the cloud and randomly draws a batch of data from it to optimize the parameters of the local network. A population-guided policy search method is introduced to speed up the convergence of the DRL agent and avoid falling into the local optimum. The experimental results demonstrate that ES-DRL can reduce the average latency by up to approximately 17 percent compared to the existing DRL-based task offloading method.

EdgeRL: Reinforcement Learning-driven Deep Learning Model Inference Optimization at Edge

An Adaptive Device-Edge Co-Inference Framework Based on Soft Actor-Critic

Accelerating Deep Neural Network Tasks Through Edge-Device Adaptive Inference

Edge-Cloud Cooperation for DNN Inference Via Reinforcement Learning and Supervised Learning

Multi-Modal Deep Reinforcement Learning for Edge-Assisted Video Analytics.

Edge Intelligence: On-Demand Deep Learning Model Co-Inference with Device-Edge Synergy

Optimal Computation Resource Allocation in Energy-Efficient Edge IoT Systems with Deep Reinforcement Learning

Towards scalable and efficient Deep-RL in edge computing: A game-based partition approach

Deep Reinforcement Learning for Edge Computing and Resource Allocation in 5G Beyond

Incentive-Driven Long-term Optimization for Edge Learning by Hierarchical Reinforcement Mechanism

Lyapunov-Driven Deep Reinforcement Learning for Edge Inference Empowered by Reconfigurable Intelligent Surfaces

Edge Caching for IoT Transient Data Using Deep Reinforcement Learning

Improving QoE of Deep Neural Network Inference on Edge Devices: A Bandit Approach

An Incentive Mechanism Design for Efficient Edge Learning by Deep Reinforcement Learning Approach

Reinforcement Learning Based Energy-Efficient Collaborative Inference for Mobile Edge Computing

Online Learning for Orchestration of Inference in Multi-User End-Edge-Cloud Networks

Deep-Edge: An Efficient Framework for Deep Learning Model Update on Heterogeneous Edge

A Deep Reinforcement Learning Approach for DNN Collaborative Inference in Mobile Edge Computing

Performance Optimization of Serverless Edge Computing Function Offloading Based on Deep Reinforcement Learning.

A3C-DO: A Regional Resource Scheduling Framework Based on Deep Reinforcement Learning in Edge Scenario

EdgeAdaptor: Online Configuration Adaption, Model Selection and Resource Provisioning for Edge DNN Inference Serving at Scale