Abstract:Recently, smart cities, healthcare system, and smart vehicles have raised challenges on the capability and connectivity of state-of-the-art Internet-of-Things (IoT) devices, especially for the devices in hotspots area. Multiaccess edge computing (MEC) can enhance the ability of emerging resource-intensive IoT applications and has attracted much attention. However, due to the time-varying network environments, as well as the heterogeneous resources of network devices, it is hard to achieve stable, reliable, and real-time interactions between edge devices and their serving edge servers, especially in the 5G ultradense network (UDN) scenarios. Ultradense edge computing (UDEC) has the potential to fill this gap, especially in the 5G era, but it still faces challenges in its current solutions, such as the lack of: 1) efficient utilization of multiple 5G resources (e.g., computation, communication, storage, and service resources); 2) low overhead offloading decision making and resource allocation strategies; and 3) privacy and security protection schemes. Thus, we first propose an intelligent UDEC (I-UDEC) framework, which integrates blockchain and artificial intelligence (AI) into 5G UDEC networks. Then, in order to achieve real-time and low overhead computation offloading decisions and resource allocation strategies, we design a novel two-timescale deep reinforcement learning (2Ts-DRL) approach, consisting of a fast-timescale and a slow-timescale learning process, respectively. The primary objective is to minimize the total offloading delay and network resource usage by jointly optimizing computation offloading, resource allocation, and service caching placement. We also leverage federated learning (FL) to train the 2Ts-DRL model in a distributed manner, aiming to protect the edge devices' data privacy. Simulation results corroborate the effectiveness of both the 2Ts-DRL and FL in the I-UDEC framework and prove that our proposed algorithm can reduce task execution time up to 31.87%.

A Double-Timescale Reinforcement Learning Based Cloud-Edge Collaborative Framework for Decomposable Intelligent Services in Industrial Internet of Things

Cloud–Edge Collaborative SFC Mapping for Industrial IoT Using Deep Reinforcement Learning

Deep Reinforcement Learning for Scheduling in an Edge Computing-Based Industrial Internet of Things

Energy-efficient Edge-cloud Collaborative Intelligent Computing: A Two-timescale Approach

Collaborative Cloud-Edge Service Cognition Framework for DNN Configuration toward Smart IIoT

Collaborative on-demand dynamic deployment via deep reinforcement learning for IoV service in multi edge clouds

Dual-Timescale Resource Allocation for Collaborative Service Caching and Computation Offloading in IoT Systems.

Edge Dynamic Service Offloading Based on Multi-agent Deep Q Learning.

Digital Twin-assisted Reinforcement Learning for Resource-aware Microservice Offloading in Edge Computing

Reinforcement Learning for Task Placement in Collaborative Cloud- Edge Computing

Deep Reinforcement Learning-based Joint Caching and Computing Edge Service Placement for Sensing-Data-Driven IIoT Applications

When Deep Reinforcement Learning Meets Federated Learning: Intelligent Multitimescale Resource Management for Multiaccess Edge Computing in 5G Ultradense Network

Delay Constrained SFC Orchestration for Edge Intelligence-Enabled IIoT: A DRL Approach

Deep Reinforcement Learning Based Service Migration Strategy for Edge Computing

When Deep Reinforcement Learning Meets Federated Learning: Intelligent Multi-Timescale Resource Management for Multi-access Edge Computing in 5G Ultra Dense Network

Edge QoE: Computation Offloading With Deep Reinforcement Learning for Internet of Things

Intelligent Transportation Vehicle Road Collaboration and Task Scheduling Based on Deep Learning in Augmented Internet of Things

SFC Embedding Meets Machine Learning: Deep Reinforcement Learning Approaches.

Deep Reinforcement Learning for Edge Service Placement in Softwarized Industrial Cyber-Physical System

Deep Reinforcement Learning-Based Resource Allocation for Content Distribution in IoT-Edge-Cloud Computing Environments

Distributed Reinforcement Learning-Based Memory Allocation for Edge-Plcs in Industrial IoT