Abstract:With the rapid development of edge-cloud computing, distributing resources to edge nodes and terminal devices to provide high-quality services for latency-sensitive applications and reduce network communication costs has become increasingly important. However, the complexity and heterogeneity of the edge-cloud environment pose significant challenges to the reliability of data storage and device load balancing. To address these issues, in this paper, we propose a Deep Reinforcement Learning (DRL)-based data replica placement scheme, BRPS. This scheme considers the different geographic locations of hardware devices, the heterogeneous storage capacity and reliability, and the different data requirements of varying user services in edge-cloud environments. Firstly, we constructed a model for data replica placement in the edge-cloud environment, addressing key factors such as latency, reliability, and load, focusing on the heterogeneity of device resources and reliability, and the diverse data needs of user services. Furthermore, we propose the BRPS scheme using the Double Deep Q-Network (DDQN) method of DRL, transforming the data replica placement issue into a multi-objective optimization problem. By placing the DRL-based decision process in a separate management edge node, the separation of decision and execution is achieved, which enhances the efficiency of data replica placement, ensures data reliability, reduces latency, and achieves system load balance. Experimental results demonstrate that BRPS significantly outperforms the existing comparison schemes while ensuring data reliability. The BRPS scheme reduces latency by 8.12% compared to the Random scheme and outperforms the best heuristic in system load balancing and memory utilization by 13.15% and 11.91%, respectively. Moreover, BRPS shows superior adaptability in extreme network congestion scenarios and effectively adapts to the dynamic changes of nodes in edge-cloud environments.

A Meta Reinforcement Learning-based Scheme for Adaptive Service Placement in Edge Computing

A Dynamic Service Placement Based on Deep Reinforcement Learning in Mobile Edge Computing

Deep Reinforcement Learning based Approach for Online Service Placement and Computation Resource Allocation in Edge Computing

A Multi-Agent Reinforcement Learning Scheme for SFC Placement in Edge Computing Networks

Deep Reinforcement Learning-Based Workload Scheduling for Edge Computing

Efficient Microservice Deployment in the Edge-Cloud Networks With Policy-Gradient Reinforcement Learning

Reinforcement Learning Framework for Server Placement and Workload Allocation in Multi-Access Edge Computing

Service Function Chain Embedding Meets Machine Learning: Deep Reinforcement Learning Approach

A Deep Reinforcement Learning Approach to Multi-Component Job Scheduling in Edge Computing

Deep reinforcement learning-based resource scheduling for energy optimization and load balancing in SDN-driven edge computing

A Balanced and Reliable Data Replica Placement Scheme Based on Reinforcement Learning in Edge-Cloud Environments

An Edge Server Placement Method Based on Reinforcement Learning

Data-Intensive Application Deployment at Edge: A Deep Reinforcement Learning Approach

DRLD-SP: A Deep Reinforcement Learning-based Dynamic Service Placement in Edge-Enabled Internet of Vehicles

A Meta Reinforcement Learning-Based Task Offloading Strategy for IoT Devices in an Edge Cloud Computing Environment

A2C-DRL: Dynamic Scheduling for Stochastic Edge-Cloud Environments Using A2C and Deep Reinforcement Learning

Dynamic Pricing Scheme for Edge Computing Services: A Two-layer Reinforcement Learning Approach

Dynamic and intelligent edge server placement based on deep reinforcement learning in mobile edge computing

Priority-Aware Resource Allocation for RIS-assisted Mobile Edge Computing Networks: A Deep Reinforcement Learning Approach

Fast Adaptive Task Offloading in Edge Computing based on Meta Reinforcement Learning

Joint Service Migration and Resource Allocation in Edge IoT System Based on Deep Reinforcement Learning