DNN Partitioning, Task Offloading, and Resource Allocation in Dynamic Vehicular Networks: A Lyapunov-Guided Diffusion-Based Reinforcement Learning Approach

Zhang Liu,Hongyang Du,Junzhe Lin,Zhibin Gao,Lianfen Huang,Seyyedali Hosseinalipour,Dusit Niyato

2024-06-11

Abstract:The rapid advancement of Artificial Intelligence (AI) has introduced Deep Neural Network (DNN)-based tasks to the ecosystem of vehicular networks. These tasks are often computation-intensive, requiring substantial computation resources, which are beyond the capability of a single vehicle. To address this challenge, Vehicular Edge Computing (VEC) has emerged as a solution, offering computing services for DNN-based tasks through resource pooling via Vehicle-to-Vehicle/Infrastructure (V2V/V2I) communications. In this paper, we formulate the problem of joint DNN partitioning, task offloading, and resource allocation in VEC as a dynamic long-term optimization. Our objective is to minimize the DNN-based task completion time while guaranteeing the system stability over time. To this end, we first leverage a Lyapunov optimization technique to decouple the original long-term optimization with stability constraints into a per-slot deterministic problem. Afterwards, we propose a Multi-Agent Diffusion-based Deep Reinforcement Learning (MAD2RL) algorithm, incorporating the innovative use of diffusion models to determine the optimal DNN partitioning and task offloading decisions. Furthermore, we integrate convex optimization techniques into MAD2RL as a subroutine to allocate computation resources, enhancing the learning efficiency. Through simulations under real-world movement traces of vehicles, we demonstrate the superior performance of our proposed algorithm compared to existing benchmark solutions.

Machine Learning

What problem does this paper attempt to address?

The paper aims to address the issues of partitioning, offloading, and resource allocation for deep neural network (DNN) tasks in vehicular networks. Specifically, since DNN tasks are typically computation-intensive and difficult for a single vehicle to handle independently, it is necessary to utilize vehicular edge computing (VEC) technology to optimize the execution process of these tasks. The goal of the paper is to minimize the completion time of DNN-based tasks while ensuring system stability. The main issues addressed include:

1. **Optimal Partitioning of DNN Tasks**: Due to the significant differences in computational requirements and output data sizes of different layers, selecting appropriate partitioning points is crucial. Improper partitioning may result in the computation-intensive parts not being offloaded to edge nodes at a low cost.

2. **Task Offloading Decision**: Once DNN tasks are partitioned, the intermediate data needs to be offloaded to other edge nodes. However, the mobility of vehicles makes the offloading process full of uncertainties, requiring real-time offloading decisions. Existing methods based on heuristic or decomposition search algorithms are not very practical when implemented in dynamic vehicular networks.

3. **Ensuring System Stability**: Most existing studies focus on improving performance metrics such as task execution delay and energy consumption but overlook the need to ensure the reliability of system operation. Ensuring stable system operation is crucial for practical applications.

To address these issues, the paper proposes a new framework that combines Lyapunov optimization techniques, diffusion models, and deep reinforcement learning methods to achieve efficient execution of DNN tasks in dynamic vehicular environments.