Abstract:Resource allocation in Narrowband Internet of Things (NB-IoT) networks is a complex challenge due to dynamic user demands, variable channel conditions, and distance considerations. Traditional approaches often struggle to adapt to the dynamic nature of these environments. In this study, we leverage reinforcement learning (RL) to address the intricate nature of NB-IoT resource allocation. Specifically, we employ the Soft Actor-Critic (SAC) algorithm, comparing its performance against conventional RL algorithms such as Deep Q-Network (DQN) and Proximal Policy Optimization (PPO). The Soft Actor-Critic (SAC) algorithm is employed to train an agent for adaptive resource allocation, considering energy efficiency, throughput, latency, fairness, and interference constraints. The agent adeptly balances these objectives through an intricate reward structure and penalty mechanisms. Through comprehensive analysis, we present performance metrics, including total reward, energy efficiency, throughput, fairness, and latency, showcasing the efficacy of SAC when compared to DQN and PPO. Our findings underscore the efficiency of SAC in optimizing resource allocation in NB-IoT networks, offering a promising solution to the complexities inherent in such dynamic environments.Resource allocation in Narrowband Internet of Things (NB-IoT) networks presents a complex challenge due to dynamic user demands, variable channel conditions, and distance considerations. Traditional approaches often struggle to adapt to these dynamic environments. This study leverages reinforcement learning (RL), specifically the Soft Actor-Critic (SAC) algorithm, to address the intricacies of NB-IoT resource allocation. We compare SAC's performance against conventional RL algorithms, including Deep Q-Network (DQN) and Proximal Policy Optimization (PPO). The SAC algorithm is utilized to train an agent for adaptive resource allocation, focusing on energy efficiency, throughput, latency, fairness, interference constraints, recovery time, and long-term performance stability. To demonstrate the scalability and effectiveness of SAC, we conducted experiments on NB-IoT networks with varying deployment types and configurations, including standard urban and suburban, high-density urban, industrial IoT, rural and low-density, and IoT service providers. To assess generalization capability, we tested SAC across applications like smart metering, smart cities, smart agriculture, and asset tracking & management. Our comprehensive analysis demonstrates that SAC significantly outperforms DQN and PPO across multiple performance metrics. Specifically, SAC improves energy efficiency by 5.60% over PPO and 10.25% over DQN. In terms of latency, SAC achieves a marginal reduction of approximately 0.0124% compared to PPO and 0.0126% compared to DQN. SAC enhances throughput by 214.98% over PPO and 15.72% over DQN. Additionally, SAC shows a substantial increase in fairness (Jain's index), improving by 358.31% over PPO and 614.46% over DQN. SAC also demonstrates superior recovery time, improving by 18.99% over PPO and 25.07% over DQN. In both deployment scenarios and diverse IoT applications, SAC consistently achieves high total rewards, minimal fluctuations, and stable performance. Energy efficiency remains constant at 7.2 bits per Joule, and latency is approximately 0.080 s. Throughput is robust across different deployments, while fairness remains high, ensuring equitable resource allocation. Recovery times are stable, enhancing operational reliability. These results underscore SAC's efficiency and robustness in optimizing resource allocation in NB-IoT networks, presenting a promising solution to the complexities of dynamic environments.

Intelligent Bandwidth Allocation for Latency Management in NG-EPON using Reinforcement Learning Methods

Intelligent Bandwidth Allocation for Latency Management in NG-EPON Using Reinforcement Learning Method

Deep Reinforcement Learning Based Massive Access Management for Ultra-Reliable Low-Latency Communications

BiLSTM Based Reinforcement Learning for Resource Allocation and User Association in LTE-U Networks

A Novel Dynamic Bandwidth Allocation Scheme in EPONs

Reinforcement Learning Assisted Bandwidth Aware Virtual Network Resource Allocation

Reinforcement learning for bandwidth estimation and congestion control in real-time communications

AoI-Aware Resource Allocation for Smart Multi-QoS Provisioning

A Simple Reinforcement Learning Mechanism for Resource Allocation in LTE-A Networks with Markov Decision Process and Q-Learning

ReinWiFi: A Reinforcement-Learning-Based Framework for the Application-Layer QoS Optimization of WiFi Networks

Multi-Agent Deep Reinforcement Learning Based Resource Allocation for Ultra-Reliable Low-Latency Internet of Controllable Things

A Novel Dynamic Bandwidth Allocation Scheme Based on International Standard in Ethernet PON

A Fair Dynamic Bandwidth Allocation Algorithm for EPON with Service Level Agreement Supporting

A Novel Upstream Dynamic Bandwidth Assignment Scheme for GPONs

Intelligent O-RAN Traffic Steering for URLLC Through Deep Reinforcement Learning

Next-gen resource optimization in NB-IoT networks: Harnessing soft actor-critic reinforcement learning

Intelligent Access to Unlicensed Spectrum: A Mean Field Based Deep Reinforcement Learning Approach

Enhancing Routing in SD-EONs through Reinforcement Learning: A Comparative Analysis

Design of dynamic resource allocation scheme for real-time traffic in the LTE network

Deep Reinforcement Learning Based Joint Allocation Scheme in a TWDM-PON-Based mMIMO Fronthaul Network

An Intelligent SDWN Routing Algorithm Based on Network Situational Awareness and Deep Reinforcement Learning