Abstract:Opportunistic computation offloading is an effective method to improve the computation performance of mobile-edge computing (MEC) networks under dynamic edge environment. In this paper, we consider a multi-user MEC network with time-varying wireless channels and stochastic user task data arrivals in sequential time frames. In particular, we aim to design an online computation offloading algorithm to maximize the network data processing capability subject to the long-term data queue stability and average power constraints. The online algorithm is practical in the sense that the decisions for each time frame are made without the assumption of knowing the future realizations of random channel conditions and data arrivals. We formulate the problem as a multi-stage stochastic mixed integer non-linear programming (MINLP) problem that jointly determines the binary offloading (each user computes the task either locally or at the edge server) and system resource allocation decisions in sequential time frames. To address the coupling in the decisions of different time frames, we propose a novel framework, named LyDROO, that combines the advantages of Lyapunov optimization and deep reinforcement learning (DRL). Specifically, LyDROO first applies Lyapunov optimization to decouple the multi-stage stochastic MINLP into deterministic per-frame MINLP subproblems. By doing so, it guarantees to satisfy all the long-term constraints by solving the per-frame subproblems that are much smaller in size. Then, LyDROO integrates model-based optimization and model-free DRL to solve the per-frame MINLP problems with very low computational complexity. Simulation results show that under various network setups, the proposed LyDROO achieves optimal computation performance while stabilizing all queues in the system. Besides, it induces very low computation time that is particularly suitable for real-time implementation in fast fading environments.

Can Online Learning Make Extreme Mobility More Reliable?

Can Online Learning Increase the Reliability of Extreme Mobility Management?

Bandit Policies for Reliable Cellular Network Handovers in Extreme Mobility

Stochastic Geometry Based Handover Probability Analysis in Dense Cellular Networks

Mobility Management in 5G and Beyond: A Novel Smart Handover with Adaptive Time-to-Trigger and Hysteresis Margin

Deep Reinforcement Learning Based Massive Access Management for Ultra-Reliable Low-Latency Communications

Online Learning Offloading Framework for Heterogeneous Mobile Edge Computing System

Deep Learning-Based Intelligent Dual Connectivity for Mobility Management in Dense Network.

Exploiting Mobility Patterns for Inter-Technology Handover in Mobile Environments

Context-Aware Online Offloading Strategy with Mobility Prediction for Mobile Edge Computing.

A Non-stochastic Learning Approach to Energy Efficient Mobility Management

Movement-Based Reliable Mobility Management for Beyond 5G Cellular Networks.

Resource Allocation in URLLC with Online Learning for Mobile Users.

Deep-Mobility: A Deep Learning Approach for an Efficient and Reliable 5G Handover

Deep Reinforcement Learning for Mobile Video Offloading in Heterogeneous Cellular Networks

A Deep Reinforcement Learning Approach for Online Computation Offloading in Mobile Edge Computing

Optimal Handover Policy for Mmwave Cellular Networks: A Multi-Armed Bandit Approach

Dynamic Fuzzy Q-learning for Handover Parameters Optimization in 5G Multi-Tier Networks

Online Learning Enabled Task Offloading for Vehicular Edge Computing.

Lyapunov-Guided Deep Reinforcement Learning for Stable Online Computation Offloading in Mobile-Edge Computing Networks

An Efficient Online Computation Offloading Approach for Large-Scale Mobile Edge Computing via Deep Reinforcement Learning