Abstract:Fog computing reduces network latency by moving computational resources close to where the data is generated. Vehicular fog computing (VFC) is an emerging computing paradigm where fog nodes deployed on moving vehicles (i.e., vehicular fog nodes (VFNs)) complement stationary fog nodes (e.g., the ones co-located with cellular base stations) to satisfy the spatiotemporally varying demand for computing resources in a cost-efficient manner. On-demand VFC (ODVFC) supports the dynamic routing of VFNs to the places where demand emerges. Different from previous works on capacity planning and vehicle routing that utilize compute-intensive optimization methods such as integer linear programming (ILP), this paper explores the feasibility of applying reinforcement learning to dynamic capacity planning in a time-efficient manner. Specifically, we propose to apply the multi-agent reinforcement learning (MARL) method (i.e., actor-critic) following the centralized training decentralized execution scheme to learn the VFN routing policies. This approach allows distributed VFNs to cooperatively maximize the techno-economic performance of ODVFC. For evaluation, we built an open-source VFC simulation platform that integrates vehicular traffic simulation with 5G NR V2X and MARL environment. Compared with decentralized learning (i.e., each VFN independently learns its routing policy), centralized learning (i.e., using a global agent for VFN routing), and ILP methods, our proposal proves to achieve 8.3% higher revenue and 13.2% higher number of served tasks than decentralized training; and it has 40.6% and 83% lower execution time than centralized learning and ILP, respectively, with only 14% lower revenue than both. It is also scalable to real-life scenarios with a great number of users and VFNs.

Cross-Regional Task Offloading with Multi-Agent Reinforcement Learning for Hierarchical Vehicular Fog Computing.

Hierarchical Task Offloading for Vehicular Fog Computing Based on Multi-Agent Deep Reinforcement Learning

Many-to-Many Task Offloading in Vehicular Fog Computing: A Multi-Agent Deep Reinforcement Learning Approach

Priority-Aware Task Offloading in Vehicular Fog Computing Based on Deep Reinforcement Learning

Dynamic Many-to-Many Task Offloading in Vehicular Fog Computing: A Multi-Agent DRL Approach

Adaptive Task Offloading in Vehicular Edge Computing Networks: a Reinforcement Learning Based Scheme

Federated Deep Reinforcement Learning-Based Task Allocation in Vehicular Fog Computing.

Mean-field reinforcement learning for decentralized task offloading in vehicular edge computing

Adaptive Task Offloading in Vehicular Edge Computing Networks Based on Deep Reinforcement Learning

Deep Reinforcement Learning-Based V2V Partial Computation Offloading in Vehicular Fog Computing

Cloud–Fog Collaborative Computing Based Task Offloading Strategy in Internet of Vehicles

DRL-based Task and Computational Offloading for Internet of Vehicles in Decentralized Computing

Learning-Based Task Offloading for Vehicular Cloud Computing Systems

Federated Multi-Agent Actor-Critic Learning Task Offloading in Intelligent Logistics

Mobility Prediction-Based Joint Task Assignment and Resource Allocation in Vehicular Fog Computing.

Task Collaborative Offloading Scheme in Vehicle Multi-Access Edge Computing Network

Task offloading and resource allocation in NOMA-VEC: A multi-agent deep graph reinforcement learning algorithm

Multi-agent Reinforcement Learning-based Capacity Planning for On-demand Vehicular Fog Computing

Competition-Awareness Partial Task Offloading and UAV Deployment for Multitier Parallel Computational Internet of Vehicles

Joint computation offloading and resource allocation for end-edge collaboration in internet of vehicles via multi-agent reinforcement learning

Leveraging Time-Critical Computation and AI Techniques for Task Offloading in Internet of Vehicles Network Applications