Muscle membrane potential, resistance, and external potassium chloride.

H. Jenerick

DOI: https://doi.org/10.1002/JCP.1030420309

1953-12-01

Journal of Cellular and Comparative Physiology

Abstract:

What problem does this paper attempt to address?

Priority/Demand-Based Resource Management with Intelligent O-RAN for Energy-Aware Industrial Internet of Things

Seyha Ros,Seungwoo Kang,Inseok Song,Geonho Cha,Prohim Tam,Seokhoon Kim

DOI: https://doi.org/10.3390/pr12122674

IF: 3.5

2024-11-29

Processes

Abstract:The last decade has witnessed the explosive growth of the internet of things (IoT), demonstrating the utilization of ubiquitous sensing and computation services. Hence, the industrial IoT (IIoT) is integrated into IoT devices. IIoT is concerned with the limitation of computation and battery life. Therefore, mobile edge computing (MEC) is a paradigm that enables the proliferation of resource computing and reduces network communication latency to realize the IIoT perspective. Furthermore, an open radio access network (O-RAN) is a new architecture that adopts a MEC server to offer a provisioning framework to address energy efficiency and reduce the congestion window of IIoT. However, dynamic resource computation and continuity of task generation by IIoT lead to challenges in management and orchestration (MANO) and energy efficiency. In this article, we aim to investigate the dynamic and priority of resource management on demand. Additionally, to minimize the long-term average delay and computation resource-intensive tasks, the Markov decision problem (MDP) is conducted to solve this problem. Hence, deep reinforcement learning (DRL) is conducted to address the optimal handling policy for MEC-enabled O-RAN architectures. In this study, MDP-assisted deep q-network-based priority/demanding resource management, namely DQG-PD, has been investigated in optimizing resource management. The DQG-PD algorithm aims to solve resource management and energy efficiency in IIoT devices, which demonstrates that exploiting the deep Q-network (DQN) jointly optimizes computation and resource utilization of energy for each service request. Hence, DQN is divided into online and target networks to better adapt to a dynamic IIoT environment. Finally, our experiment shows that our work can outperform reference schemes in terms of resources, cost, energy, reliability, and average service completion ratio.

engineering, chemical
Deep Reinforcement Learning for Online Resource Allocation in IoT Networks: Technology, Development, and Future Challenges

Peng Cheng,Youjia Chen,Ming Ding,Zhuo Chen,Sige Liu,Yi-Ping Phoebe Chen

DOI: https://doi.org/10.1109/mcom.001.2200526

IF: 9.03

2023-06-20

IEEE Communications Magazine

Abstract:The growing number of complex and heterogeneous Internet of Things (IoT) applications has imposed a high demand for scarce communications and computing resources. To meet this stringent requirement, it is desirable to develop large-scale highly adaptive online resource allocation strategies to streamline existing network operations. Deep reinforcement learning (DRL), which combines the merits of reinforcement learning and deep learning, is capable of addressing complex decision-making tasks, thus enabling efficient online resource allocation. In this article, we present a DRL-based resource allocation framework. We begin a discussion on DRL basics and review its several recent applications. Then, we develop two new DRL algorithms that facilitate unlocking the potential of DRL and offer viable solutions to many more complex resource allocation problems. The first one tackles an optimization problem exposed to mixed (discrete and continuous) action spaces and bound by a number of highly non-linear quality-of-service (QoS) constraints. The second one extends the single-agent DRL to a more challenging multi-agent DRL by introducing a novel semi-distributed architecture. Finally, we discuss the challenges and future visions of applying DRL to real-world IoT networks.

telecommunications,engineering, electrical & electronic
QDRL: Queue-Aware Online DRL for Computation Offloading in Industrial Internet of Things

Aikun Xu,Zhigang Hu,Xinyu Zhang,Hui Xiao,Hao Zheng,Bolei Chen,Meiguang Zheng,Ping Zhong,Yilin Kang,Keqin Li

DOI: https://doi.org/10.1109/jiot.2023.3316139

IF: 10.6

2023-01-01

IEEE Internet of Things Journal

Abstract:Recently, the Industrial Internet of Things (IIoT) has shown great application value in environmental monitoring. However, it suffers from serious bottlenecks in energy and computing capability. To address them, researchers have made lots of effort. Nevertheless, they neglect either the edge-end collaboration or the impact of task queue backlog, resulting in low system revenue. To this end, we design a Queue-aware computation offloading method based on DRL (QDRL). Specifically, we represent the long-term system operation as a Multi-stage Stochastic Mixed-Integer optimization Problem (M-SMIP), which is further converted into a deterministic problem using Lyapunov optimization. Given that the resource allocation and computation offloading in this deterministic problem are strongly coupled and difficult to solve, we decompose this problem into two subproblems. Subsequently, a reinforcement learning scheme with Actor-Critic architecture is designed to solve these subproblems. The Actor module is designed based on a deep learning model and quantization strategy for generating computation offloading actions. The mathematical reasoning and learning-based methods are integrated as the Critic module for achieving resource allocation. Extensive simulation results show that the performance of QDRL surpasses four baselines and approaches the approximate optimal algorithm in terms of average task queue length, normalized real computation rate, and computation time.

computer science, information systems,telecommunications,engineering, electrical & electronic
DNN Deployment, Task Offloading, and Resource Allocation for Joint Task Inference in IIoT

Wenhao Fan,Zeyu Chen,Zhibo Hao,Yi Su,Fan Wu,Bihua Tang,Yuan'an Liu

DOI: https://doi.org/10.1109/tii.2022.3192882

IF: 12.3

2022-12-17

IEEE Transactions on Industrial Informatics

Abstract:Joint task inference, which fully utilizes end edge cloud cooperation, can effectively enhance the performance of deep neural network (DNN) inference services in the industrial internet of things (IIoT) applications. In this paper, we propose a novel joint resource management scheme for a multi task and multi service scenario consisting of multiple sensors, a cloud server, and a base station equipped with an edge server . A time slotted system model is proposed, incorporating DNN deployment, data size control, task offloading, computing resource allocation, and wireless channel allocation. Among them, the DNN deployment is to deploy proper DNNs on the edge server under its total resource constraint, and the data size control is to make trade off between task inference accuracy and task transmission delay through changing task da ta size. Our goal is to minimize the total cost including total task processing delay and total error inference penalty while guaranteeing long term task queue stability and all task inference accuracy requirements. Leveraging the Lyapunov optimization, we first transform the optimization problem into a deterministic problem for each time slot. Then, a deep deterministic policy gradient (DDPG) based deep reinforcement learning (DRL) algorithm is designed to provide the near optimal solution. We further desi gn a fast numerical method for the data size control sub problem to reduce the training complexity of the DRL model, and design a penalty mechanism to prevent frequent optimizations of DNN deployment. Extensive experiments are conducted by varying differen t crucial parameters. The superiority of our scheme is demonstrated in comparison with 3 other schemes.

automation & control systems,computer science, interdisciplinary applications,engineering, industrial
AoI-Aware Resource Scheduling for Industrial IoT with Deep Reinforcement Learning

Hongzhi Li,Lin Tang,Shengwei Chen,Libin Zheng,Shaohong Zhong

DOI: https://doi.org/10.3390/electronics13061104

IF: 2.9

2024-03-18

Electronics

Abstract:Effective resource scheduling methods in certain scenarios of Industrial Internet of Things are pivotal. In time-sensitive scenarios, Age of Information is a critical indicator for measuring the freshness of data. This paper considers a densely deployed time-sensitive Industrial Internet of Things scenario. The industrial wireless device transmits data packets to the base station with limited channel resources under the constraints of Age of Information. It is assumed that each device has the capacity to store the packets it generates. The device will discard the data to alleviate the data queue backlog when the Age of Information of the data packet exceeds the threshold. We developed a new system utility equation to represent the scheduling problem and the problem is expressed as a trade-off between minimizing the average Age of Information and maximizing network throughput. Inspired by the success of reinforcement learning in decision-processing problems, we attempt to obtain an optimal scheduling strategy via deep reinforcement learning. In addition, a reward function is constructed to enable the agent to achieve improved convergence results. Compared with the baseline, our proposed algorithm can achieve better system utility and lower Age of Information violation rate.

engineering, electrical & electronic,computer science, information systems,physics, applied
Energy‐efficient resource allocation over wireless communication systems through deep reinforcement learning

Kirti Shukla,Archana Kollu,Poonam Panwar,Mukesh Soni,Latika Jindal,Hemlata Patel,Ismail Keshta,Renato R. Maaliw

DOI: https://doi.org/10.1002/dac.5589

2023-08-23

International Journal of Communication Systems

Abstract:This study introduces a deep reinforcement learning energy allocation strategy for Internet of Things terminal devices. It employs the DQN algorithm to interact with the environment and resolve the energy shortage issue in the presence of renewable energy and unpredictability. Summary As the popularity of the Internet of Things (IoT) increases, so do the energy requirements of IoT terminal equipment. To address the energy shortage problem of equipment and ensure continuous and stable operation in light of renewable energy and an uncertain environment, a rational and efficient energy allocation strategy is required. This paper proposes a deep reinforcement learning energy allocation strategy that uses the DQN algorithm to directly interact with the unknown environment. The best energy allocation method is independent of environmental knowledge, and a pretraining algorithm is proposed to maximise the initialization state of the strategy. Experiments of comparison and simulation are conducted under various channel data circumstances. Results indicate that the proposed energy allocation strategy outperforms the current strategy in multiple channel conditions and has a high capacity for adaptation to changing conditions.

telecommunications,engineering, electrical & electronic
Deep Reinforcement Learning-Based Computation Offloading and Resource Allocation in IoT

Yongli Yang,Qi Liu,Qinghua Zhu,Yong Liu,Wei Han

DOI: https://doi.org/10.1109/icaml57167.2022.00058

2022-01-01

Abstract:To address the issue of offloading decision and resource allocation, this paper proposes an approach based on the Q-learning algorithm and DQN algorithm. To start with, a system model is designed in accordance with the single marginal environment system models, including the network model and computing model. Through definition and design of the aforesaid two models, the issue of optimization and constraints facing the optimization are identified and formularized. Next, Q-learning and DQN are used to design the solution plans. Finally, simulation experiment is conducted to compare the SGN algorithm with other solution plans, and point out advantages of the DQN algorithm in mobile edge computing.
Intelligent-Driven Green Resource Allocation for Industrial Internet of Things in 5G Heterogeneous Networks

Peng Yu,Mo Yang,Ao Xiong,Yahui Ding,Wenjing Li,Xuesong Qiu,Luoming Meng,Michel Kadoch,Mohamed Cheriet

DOI: https://doi.org/10.1109/tii.2020.3041159

IF: 12.3

2022-01-01

IEEE Transactions on Industrial Informatics

Abstract:The Industrial Internet of Things (IIoT) is one of the important applications under the 5G massive machine type of communication (mMTC) scenario. To ensure the high reliability of IIoT services, it is necessary to apply an efficient resource allocation method under the dynamic and complex environment. In view of the absence of energy-efficient resource management architecture for the entire network, this article proposes an intelligent-driven green resource allocation mechanism for the IIoT under 5G heterogeneous networks. First, an intelligent end-to-end self-organizing resource allocation framework for IIoT service is given. Next, an energy-efficient resource allocation model within the framework is proposed. It is then solved by an intelligent mechanism with the asynchronous advantage actor critic driven deep reinforcement learning algorithm. Through the comparison analysis of different methods and rewards under IIoT scenarios with proper parameters setting, the proposed method can achieve better performance than other traditional deep learning (DL) methods and maintain service quality above accepted levels as well.

automation & control systems,computer science, interdisciplinary applications,engineering, industrial
Toward Computing Resource Reservation Scheduling in Industrial Internet of Things

Fan Liang,D. Griffith,N. Golmie,Xing-fa Liu,Wei Yu

DOI: https://doi.org/10.1109/JIOT.2020.3044057

IF: 10.6

2021-05-15

IEEE Internet of Things Journal

Abstract:The Industrial Internet of Things (IIoT) is a critically important implementation of the Internet of Things (IoT), connecting IoT devices ubiquitously in an industrial environment. Based on the interconnection of IoT devices, IIoT applications can collect and analyze sensing data, which help operators to control and manage manufacturing systems, leading to significant performance improvements and enabling automation. IIoT systems are characterized by a variety of IIoT applications, which generate different computing tasks depending on their functionalities. Some tasks are time sensitive (TS), while others are not, and more importantly, some tasks are nonpreemptive in IIoT scenarios. Thus, processing the different IIoT applications efficiently in an IIoT environment is key to achieving automation. Since computing resources are limited in IIoT, how to rapidly process TS tasks is a critical issue. Although some existing scheduling schemes can deal with the latency requirements of TS tasks, they lack consideration for nonpreemptive tasks. To address this issue, in this article we consider a typical smart warehouse system as an example and propose a generic task scheduling scheme that reserves computing resources to wait for upcoming TS tasks in such an IIoT environment. In doing so, our proposed scheme is capable of minimizing the overall waiting time for TS tasks. To evaluate the proposed scheme, we have implemented a simulation platform for a smart warehouse and conducted extensive experiments. Our experimental results demonstrate the efficacy of our scheme, which can allocate computing resources so that the processing time for the TS tasks can be reduced. Additionally, we discuss some potential research directions toward improving performance in IIoT environments with respect to resource management, machine learning, and security and privacy.

Computer Science,Engineering
Maximizing throughput in NOMA-enable industrial IoT networks using digital twin and reinforcement learning

Sekione Reward Jeremiah,David Camacho,Jong Hyuk Park

DOI: https://doi.org/10.1016/j.jare.2024.04.021

2024-04-21

Abstract:Introduction: Increased deployment of heterogeneous and complex Industrial Internet of Things (IIoT) applications such as predictive maintenance and asset tracking places a substantial strain on the limited computational and communication resources. To cater to the rigorous demands of these applications, it is imperative to devise an adaptive online resource allocation method to enhance the efficiency of the current network operations. Multiaccess edge computing (MEC) and digital twins (DTs) are promising solutions that facilitate the realization of edge intelligence and find applications in various industrial applications. Yet, little is known about the advantage the two technologies offer to IIoT networks. Objective: This study presents a joint optimization of offloading and resource allocation approach where MEC-server DT is created at the edge, and nonorthogonal multiple access (NOMA) communication is considered between IIoT devices and the industrial gateways (IGWs) for spectral efficiency. Our proposed framework is tailored to reduce mean task completion latency and enhance overall IIoT network throughput. Method: To achieve our objective, we jointly optimize the computation resource allocation (RA), subchannel assignment (SA), and offloading decisions (OD). Given the inherent complexity of the problem, we further divide it into RA and SA/OD sub-problems. Employing Deep Reinforcement Learning (DRL), we have formulated a solution delineating the most efficient RA strategy and leveraged DT for optimal SA/OD strategies. Results: Simulation results demonstrate the superior efficiency of our framework, realizing up to 92 % of the efficiency of the exhaustive search method while reducing computation and action decision time. Conclusion: In light of system dynamics considered for our work, the proposed framework perfomance showcase its robustness and potential application in real-world IIoT networks.
Deep Reinforcement Learning-Based Resource Allocation for Satellite Internet of Things with Diverse QoS Guarantee

Siqi Tang,Zhisong Pan,Guyu Hu,Yang Wu,Yunbo Li

DOI: https://doi.org/10.3390/s22082979

IF: 3.9

2022-04-13

Sensors

Abstract:Large-scale terminals’ various QoS requirements are key challenges confronting the resource allocation of Satellite Internet of Things (S-IoT). This paper presents a deep reinforcement learning-based online channel allocation and power control algorithm in an S-IoT uplink scenario. The intelligent agent determines the transmission channel and power simultaneously based on contextual information. Furthermore, the weighted normalized reward concerning success rate, power efficiency, and QoS requirement is adopted to balance the performance between increasing resource efficiency and meeting QoS requirements. Finally, a practical deployment mechanism based on transfer learning is proposed to promote onboard training efficiency and to reduce computation consumption of the training process. The simulation demonstrates that the proposed method can balance the success rate and power efficiency with QoS requirement guaranteed. For S-IoT’s normal operation condition, the proposed method can improve the power efficiency by 60.91% and 144.44% compared with GA and DRL_RA, while its power efficiency is only 4.55% lower than that of DRL-EERA. In addition, this method can be transferred and deployed to a space environment by merely 100 onboard training steps.

engineering, electrical & electronic,chemistry, analytical,instruments & instrumentation
DeepEdge: A New QoE-Based Resource Allocation Framework Using Deep Reinforcement Learning for Future Heterogeneous Edge-IoT Applications

Ismail Alqerm,Jianli Pan

DOI: https://doi.org/10.1109/tnsm.2021.3123959

2021-12-01

IEEE Transactions on Network and Service Management

Abstract:Edge computing is emerging to empower the future of Internet of Things (IoT) applications. However, due to heterogeneity of applications, it is a significant challenge for the edge cloud to effectively allocate multidimensional limited resources (CPU, memory, storage, bandwidth, etc.) with constraints of applications’ Quality of Service (QoS) requirements. In this paper, we address the resource allocation problem in Edge-IoT systems through developing a novel framework named DeepEdge that allocates resources to the heterogeneous IoT applications with the goal of maximizing users’ Quality of Experience (QoE). To achieve this goal, we develop a novel QoE model that considers aligning the heterogeneous requirements of IoT applications to the available edge resources. The alignment is achieved through selection of QoS requirement range that can be satisfied by the available resources. In addition, we propose a novel two-stage deep reinforcement learning (DRL) scheme that effectively allocates edge resources to serve the IoT applications and maximize the users’ QoE. Unlike the typical DRL, our scheme exploits deep neural networks (DNN) to improve actions’ exploration by using DNN to map the Edge-IoT state to joint resource allocation action that consists of resource allocation and QoS class. The joint action not only maximize users’ QoE and satisfies heterogeneous applications’ requirements but also align the QoS requirements to the available resources. In addition, we develop a Q-value approximation approach to tackle the large space problem of Edge-IoT. Further evaluation shows that DeepEdge brings considerable improvements in terms of QoE, latency and application tasks’ success ratio in comparison to the existing resource allocation schemes.

computer science, information systems
Deep-Learning-Based Wireless Resource Allocation With Application to Vehicular Networks

Le Liang,Hao Ye,Guanding Yu,Geoffrey Ye Li

DOI: https://doi.org/10.1109/JPROC.2019.2957798

IF: 20.6

2020-01-01

Proceedings of the IEEE

Abstract:It has been a long-held belief that judicious resource allocation is critical to mitigating interference, improving network efficiency, and ultimately optimizing wireless communication performance. The traditional wisdom is to explicitly formulate resource allocation as an optimization problem and then exploit mathematical programming to solve the problem to a certain level of optimality. Nonetheless, as wireless networks become increasingly diverse and complex, for example, in the high-mobility vehicular networks, the current design methodologies face significant challenges and thus call for rethinking of the traditional design philosophy. Meanwhile, deep learning, with many success stories in various disciplines, represents a promising alternative due to its remarkable power to leverage data for problem solving. In this article, we discuss the key motivations and roadblocks of using deep learning for wireless resource allocation with application to vehicular networks. We review major recent studies that mobilize the deep-learning philosophy in wireless resource allocation and achieve impressive results. We first discuss deep-learning-assisted optimization for resource allocation. We then highlight the deep reinforcement learning approach to address resource allocation problems that are difficult to handle in the traditional optimization framework. We also identify some research directions that deserve further investigation.
An trusted computing resources optimal scheduling algorithm in Industrial Internet and healthcare integrating DRL, blockchain and End-Edge-Cloud

Zongmei Liu,Jianxin Li

DOI: https://doi.org/10.1142/s0219519423400560

2023-04-12

Journal of Mechanics in Medicine and Biology

Abstract:With the rapid development of the Internet of Things (IoT) and Internet technology, the product of the combination of the two, the Industrial Internet, has also received extensive attention and there are more and more research achievements related to the Industrial Internet. In the industrial Internet system, the communication network system composed of sensors, communication nodes, controllers and other intelligent devices can realize efficient and convenient data interaction between people and machines, providing an important infrastructure and technical support for industrial big data analysis and intelligent production. However, in the current industrial Internet system, industrial equipment users generally have the problem of low computing energy efficiency, and the collected industrial data has a high-security risk in the transmission, processing and other processes. At the same time, the size and scale of the industrial Internet equipment group is huge, and the lack of rational resource allocation leads to excessive waste of computing resources in the system, which is also a prominent problem of the current industrial Internet system. In response to the above questions, this paper, on the basis of reading a large number of documents, integrates the improved DRL algorithm, End-Edge-Cloud architecture and blockchain to form a new industrial Internet architecture. The architecture realizes computing offload through the three-tier structure of terminal layer, edge layer and cloud layer, and guarantees the security of industrial data through the decentralized feature of blockchain, ultimately achieving the goal of reducing energy consumption, computing overhead and trusted computing. In the architecture proposed in this paper, the dynamic unloading of industrial data and computing tasks is achieved through a three-tier architecture. The MDP is used to build an optimization problem model, and the improved DRL algorithm is used to iteratively solve the optimal computing resource scheduling strategy. The main research contents of this paper include (1) Using MDP to model optimization problems; (2) Propose an industrial Internet system architecture that integrates and improves DRL, “end edge cloud” and blockchain; (3) The MDP problem is solved iteratively based on deep reinforcement learning. The simulation results show that the proposed architecture has more advantages than the existing six architectures in terms of computing cost, equipment energy consumption and total working time.

engineering, biomedical,biophysics
Deep Q-Network Based Resource Allocation for UAV-assisted Ultra-Dense Networks

Xin Chen,Xu Liu,Ying Chen,Libo Jiao,Geyong Min

DOI: https://doi.org/10.1016/j.comnet.2021.108249

IF: 5.493

2021-01-01

Computer Networks

Abstract:With the rapid development of the fifth-generation (5G) wireless communications, the number of users is increasing dramatically and Ultra-Dense Networks (UDN) are becoming more important for supporting numerous users and emerging mission-critical applications. In order to conquer the communication restrictions caused by natural disasters, an emergency communication system using Unmanned Aerial Vehicles (UAV) as a flying base station (BS) to assist UDN is proposed. By virtue of the resource allocation scheme of UAV-assisted UDN systems, communication resources can be reasonably and effectively allocated to improve the quality of user experience. Firstly, aiming to maximize the system energy efficiency (EE), a UDN system model including the BS selection is constructed. Secondly, Markov Decision Process (MDP) theory is applied to transform the system model into a stochastic optimization problem. Finally, by using deep reinforcement learning (DRL) technique, we propose a Deep Q-Network (DQN) based resource allocation scheme to maximize the system energy efficiency. Simulation results exhibit that the proposed DQN-based resource allocation scheme can significantly improve the system EE compared with the legacy Q-Learning, random and maximum resource allocation algorithms.
AI-Based Resource Allocation Techniques in Wireless Sensor Internet of Things Networks in Energy Efficiency with Data Optimization

Quazi Warisha Ahmed,Shruti Garg,Amrita Rai,Manikandan Ramachandran,Noor Zaman Jhanjhi,Mehedi Masud,Mohammed Baz

DOI: https://doi.org/10.3390/electronics11132071

IF: 2.9

2022-07-01

Electronics

Abstract:For the past few years, the IoT (Internet of Things)-based restricted WSN (Wireless sensor network) has sparked a lot of attention and progress in order to attain improved resource utilisation as well as service delivery. For data transfer between heterogeneous devices, IoT requires a stronger communication network and an ideally placed energy-efficient WSN. This study uses deep learning architectures to provide a unique resource allocation method for wireless sensor IoT networks with energy efficiency as well as data optimization. EE (Energy efficiency) and SE (spectral efficiency) are two competing optimization goals in this case. The network's energy efficiency has been improved because of a deep neural network based on whale optimization. The heuristic-based multi-objective firefly algorithm was used to optimise the data. This proposed method is applied to optimal power allocation and relay selection. The study is for a cooperative multi-hop network topology. The best resource allocation is achieved by reducing overall transmit power, and the best relay selection is accomplished by meeting Quality of Service (QoS) standards. As a result, an energy-efficient protocol has been created. The simulation results demonstrate the suggested model's competitive performance when compared to traditional models in terms of throughput of 96%, energy efficiency of 95%, QoS of 75%, spectrum efficiency of 85%, and network lifetime of 91 percent.

engineering, electrical & electronic,computer science, information systems,physics, applied
Inner External DQN LoRa SF Allocation Scheme for Complex Environments

Shengli Pang,Delin Kong,Xute Wang,Ruoyu Pan,Honggang Wang,Zhifan Ye,Di Liu

DOI: https://doi.org/10.3390/electronics13142761

IF: 2.9

2024-07-15

Electronics

Abstract:In recent years, with the development of Internet of Things technology, the demand for low-power wireless communication technology has been growing, giving rise to LoRa technology. A LoRa network mainly consists of terminal nodes, gateways, and LoRa network servers. As LoRa networks often deploy many terminal node devices for environmental sensing, the limited resources of LoRa technology, the explosive growth in the number of nodes, and the ever-changing complex environment pose unprecedented challenges for the performance of the LoRa network. Although some research has already addressed the challenges by allocating channels to the LoRa network, the impact of complex and changing environmental factors on the LoRa network has yet to be considered. Reasonable channel allocation should be tailored to the situation and should face different environments and network distribution conditions through continuous adaptive learning to obtain the corresponding allocation strategy. Secondly, most of the current research only focuses on the channel adjustment of the LoRa node itself. Still, it does not consider the indirect impact of the node's allocation on the entire network. The Inner External DQN SF allocation method (IEDQN) proposed in this paper improves the packet reception rate of the whole system by using reinforcement learning methods for adaptive learning of the environment. It considers the impact on the entire network of the current node parameter configuration through nested reinforcement learning for further optimization to optimize the whole network's performance. Finally, this paper evaluates the performance of IEDQN through simulation. The experimental results show that the IEDQN method optimizes network performance.

engineering, electrical & electronic,physics, applied,computer science, information systems
Resource Allocation Using Gradient Boosting Aided Deep Q-Network for IoT in C-RANs

Yifan Luo,Jiawei Yang,Wei Xu,Kezhi Wang,Marco Di Renzo

DOI: https://doi.org/10.48550/arXiv.1910.13084

2019-10-29

Abstract:In this paper, we investigate dynamic resource allocation (DRA) problems for Internet of Things (IoT) in real-time cloud radio access networks (C-RANs), by combining gradient boosting approximation and deep reinforcement learning to solve the following two major problems. Firstly, in C-RANs, the decision making process of resource allocation is time-consuming and computational-expensive, motivating us to use an approximation method, i.e. the gradient boosting decision tree (GBDT) to approximate the solutions of second order cone programming (SOCP) problem. Moreover, considering the innumerable states in real-time C-RAN systems, we employ a deep reinforcement learning framework, i.e., deep Q-network (DQN) to generate a robust policy that controls the status of remote radio heads (RRHs). We propose a GBDT-based DQN framework for the DRA problem, where the heavy computation to solve SOCP problems is cut down and great power consumption is saved in the whole C-RAN system. We demonstrate that the generated policy is error-tolerant even the gradient boosting regression may not be strictly subject to the constraints of the original problem. Comparisons between the proposed method and existing baseline methods confirm the advantages of our method.

Information Theory
Deep Reinforcement Learning-Based Power Allocation for Minimizing Age of Information and Energy Consumption in Multi-Input Multi-Output and Non-Orthogonal Multiple Access Internet of Things Systems

Qiong Wu,Zheng Zhang,Hongbiao Zhu,Pingyi Fan,Qiang Fan,Huiling Zhu,Jiangzhou Wang

DOI: https://doi.org/10.3390/s23249687

IF: 3.9

2023-12-08

Sensors

Abstract:Multi-input multi-output and non-orthogonal multiple access (MIMO-NOMA) Internet-of-Things (IoT) systems can improve channel capacity and spectrum efficiency distinctly to support real-time applications. Age of information (AoI) plays a crucial role in real-time applications as it determines the timeliness of the extracted information. In MIMO-NOMA IoT systems, the base station (BS) determines the sample collection commands and allocates the transmit power for each IoT device. Each device determines whether to sample data according to the sample collection commands and adopts the allocated power to transmit the sampled data to the BS over the MIMO-NOMA channel. Afterwards, the BS employs the successive interference cancellation (SIC) technique to decode the signal of the data transmitted by each device. The sample collection commands and power allocation may affect the AoI and energy consumption of the system. Optimizing the sample collection commands and power allocation is essential for minimizing both AoI and energy consumption in MIMO-NOMA IoT systems. In this paper, we propose the optimal power allocation to achieve it based on deep reinforcement learning (DRL). Simulations have demonstrated that the optimal power allocation effectively achieves lower AoI and energy consumption compared to other algorithms. Overall, the reward is reduced by 6.44% and 11.78% compared the to GA algorithm and random algorithm, respectively.

engineering, electrical & electronic,chemistry, analytical,instruments & instrumentation

Muscle membrane potential, resistance, and external potassium chloride.

Priority/Demand-Based Resource Management with Intelligent O-RAN for Energy-Aware Industrial Internet of Things

Deep Reinforcement Learning for Online Resource Allocation in IoT Networks: Technology, Development, and Future Challenges

QDRL: Queue-Aware Online DRL for Computation Offloading in Industrial Internet of Things

DNN Deployment, Task Offloading, and Resource Allocation for Joint Task Inference in IIoT

AoI-Aware Resource Scheduling for Industrial IoT with Deep Reinforcement Learning

Energy‐efficient resource allocation over wireless communication systems through deep reinforcement learning

Deep Reinforcement Learning-Based Computation Offloading and Resource Allocation in IoT

Intelligent-Driven Green Resource Allocation for Industrial Internet of Things in 5G Heterogeneous Networks

Toward Computing Resource Reservation Scheduling in Industrial Internet of Things

Maximizing throughput in NOMA-enable industrial IoT networks using digital twin and reinforcement learning

Deep Reinforcement Learning-Based Resource Allocation for Satellite Internet of Things with Diverse QoS Guarantee

DeepEdge: A New QoE-Based Resource Allocation Framework Using Deep Reinforcement Learning for Future Heterogeneous Edge-IoT Applications

Deep-Learning-Based Wireless Resource Allocation With Application to Vehicular Networks

An trusted computing resources optimal scheduling algorithm in Industrial Internet and healthcare integrating DRL, blockchain and End-Edge-Cloud

Deep Q-Network Based Resource Allocation for UAV-assisted Ultra-Dense Networks

AI-Based Resource Allocation Techniques in Wireless Sensor Internet of Things Networks in Energy Efficiency with Data Optimization

Inner External DQN LoRa SF Allocation Scheme for Complex Environments

Resource Allocation Using Gradient Boosting Aided Deep Q-Network for IoT in C-RANs

Deep Reinforcement Learning-Based Power Allocation for Minimizing Age of Information and Energy Consumption in Multi-Input Multi-Output and Non-Orthogonal Multiple Access Internet of Things Systems