Abstract:The cloud-edge-terminal collaborative network (CETCN) is considered as a novel paradigm for emerging applications owing to its huge potential in providing low-latency and ultra-reliable computing services. However, achieving such benefits is very challenging due to the heterogeneous computing power of terminal devices and the complex environment faced by the CETCN. In particular, the high-dimensional and dynamic environment states cause difficulties for the CETCN to make efficient decisions in terms of task offloading, collaborative caching and mobility management. To this end, artificial intelligence (AI), especially deep reinforcement learning (DRL) has been proven effective in solving sequential decision-making problems in various domains, and offers a promising solution for the above-mentioned issues due to several reasons. Firstly, accurate modelling of the CETCN, which is difficult to obtain for real-world applications, is not required for the DRL-based method. Secondly, DRL can effectively respond to high-dimensional and dynamic tasks through iterative interactions with the environment. Thirdly, due to the complexity of tasks and the differences in resource supply among different vendors, collaboration is required between different vendors to complete tasks. The multi-agent DRL (MADRL) methods are very effective in solving collaborative tasks, where the collaborative tasks can be jointly completed by cloud, edge and terminal devices which provided by different vendors. This survey provides a comprehensive overview regarding the applications of DRL and MADRL in the context of CETCN. The first part of this survey provides a depth overview of the key concepts of the CETCN and the mathematical underpinnings of both DRL and MADRL. Then, we highlight the applications of RL algorithms in solving various challenges within CETCN, such as task offloading, resource allocation, caching and mobility management. In addition, we extend discussion to explore how DRL and MADRL are making inroads into emerging CETCN scenarios like intelligent transportation system (ITS), the industrial Internet of Things (IIoT), smart health and digital agriculture. Furthermore, security considerations related to the application of DRL within CETCN are addressed, along with an overview of existing standards that pertain to edge intelligence. Finally, we list several lessons learned in this evolving field and outline future research opportunities and challenges that are critical for the development of the CETCN. We hope this survey will attract more researchers to investigate scalable and decentralized AI algorithms for the design of CETCN.

AI-Enhanced Cloud-Edge-Terminal Collaborative Network: Survey, Applications, and Future Directions

Federated Deep Reinforcement Learning-enabled Task Offloading in Cloud-Edge-Terminal Collaborative Networks

RL/DRL Meets Vehicular Task Offloading Using Edge and Vehicular Cloudlet: A Survey

Toward Enhanced Reinforcement Learning-Based Resource Management via Digital Twin: Opportunities, Applications, and Challenges

Deep Reinforcement Learning for Computation and Communication Resource Allocation in Multiaccess MEC Assisted Railway IoT Networks

Deep Graph Reinforcement Learning for Mobile Edge Computing: Challenges and Solutions

Machine and Deep Learning for Resource Allocation in Multi-Access Edge Computing: A Survey

Beyond the Edge: An Advanced Exploration of Reinforcement Learning for Mobile Edge Computing, its Applications, and Future Research Trajectories

RLCS: Towards a robust and efficient mobile edge computing resource scheduling and task offloading system based on graph neural network

Deep Reinforcement Learning Aided Computation Offloading and Resource Allocation for IoT

Deep Reinforcement Learning for Online Resource Allocation in IoT Networks: Technology, Development, and Future Challenges

Integrated Computing, Caching, and Communication for Trust-Based Social Networks - A Big Data DRL Approach.

Cloud-Edge-End Collaborative Task Offloading in Vehicular Edge Networks: A Multi-Layer Deep Reinforcement Learning Approach

Deep Reinforcement Learning-Based Traffic Engineering in Cloud-Edge-End Collaboration Environments

Intelligent Transportation Vehicle Road Collaboration and Task Scheduling Based on Deep Learning in Augmented Internet of Things

Cloud–Edge Collaborative Resource Allocation for Blockchain-Enabled Internet of Things: A Collective Reinforcement Learning Approach

When Deep Reinforcement Learning Meets Federated Learning: Intelligent Multitimescale Resource Management for Multiaccess Edge Computing in 5G Ultradense Network

Cloud-Edge–End Collaborative Task Offloading in Vehicular Edge Networks: A Multilayer Deep Reinforcement Learning Approach

Survey on Reinforcement Learning Applications in Communication Networks.

Multi-user Resource Control with Deep Reinforcement Learning in IoT Edge Computing

Distributed Resource Scheduling for Large-Scale MEC Systems: A Multiagent Ensemble Deep Reinforcement Learning With Imitation Acceleration