Deep Reinforcement Learning-Based Mining Task Offloading Scheme for Intelligent Connected Vehicles in UAV-Aided MEC

Chunlin Li,Kun Jiang,Yong Zhang,Lincheng Jiang,Youlong Luo,Shaohua Wan

DOI: https://doi.org/10.1145/3653451

IF: 1.447

2024-03-20

ACM Transactions on Design Automation of Electronic Systems

Abstract:The convergence of unmanned aerial vehicle (UAV)-aided mobile edge computing (MEC) networks and blockchain transforms the existing mobile networking paradigm. However, in the temporary hotspot scenario for intelligent connected vehicles (ICVs) in UAV-aided MEC networks, deploying blockchain-based services and applications in vehicles is generally impossible due to its high computational resource and storage requirements. One possible solution is to offload part of all the computational tasks to MEC servers wherever possible. Unfortunately, due to the limited availability and high mobility of the vehicles, there is still lacking simple solutions that can support low-latency and higher reliability networking services for ICVs. In this paper, we study the task offloading problem of minimizing the total system latency and the optimal task offloading scheme, subject to constraints on the hover position coordinates of the UAV, the fixed bonuses, flexible transaction fees, transaction rates, mining difficulty, costs and battery energy consumption of the UAV. The problem is confirmed to be a challenging linear integer planning problem, we formulate the problem as a constrained Markov decision process (CMDP). Deep Reinforcement Learning (DRL) has excellently solved sequential decision-making problems in dynamic ICVs environment, therefore, we propose a novel distributed DRL-based P-D3QN approach by using Prioritized Experience Replay (PER) strategy and the dueling double deep Q-network (D3QN) algorithm to solve the optimal task offloading policy effectively. Finally, experiment results show that compared with the benchmark scheme, the P-D3QN algorithm can bring about 26.24% latency improvement and increase about 42.26% offloading utility.

computer science, software engineering, hardware & architecture

What problem does this paper attempt to address?

The paper aims to address the challenges faced by Intelligent Connected Vehicles (ICVs) when performing blockchain mining tasks in temporary hotspot areas within UAV-aided Mobile Edge Computing (MEC) networks. Specifically, due to the high computational and storage demands of blockchain services and applications, direct deployment on vehicles becomes impractical. Therefore, the paper proposes a task offloading scheme based on Deep Reinforcement Learning (DRL) to optimize the offloading decisions of mining tasks, minimizing the overall system delay and improving the utility of task offloading.

The main objectives include:
1. Efficiently offloading mining tasks in a complex and dynamic UAV-aided MEC environment.
2. Achieving partial offloading during the process, utilizing idle computational resources to reduce system delay.
3. Maximizing mining revenue and offloading utility in resource-constrained ICV scenarios.

By constructing a UAV-aided MEC model integrated with blockchain technology and formalizing the problem as a Constrained Markov Decision Process (CMDP), the authors propose a new distributed DRL algorithm—P-D3QN method, which leverages the Prioritized Experience Replay (PER) strategy and the Dueling Double Deep Q-Network (D3QN) algorithm to effectively solve the optimal task offloading strategy problem. Experimental results show that, compared to benchmark algorithms, this algorithm can significantly reduce delay and increase offloading utility.