Abstract:In this paper, the problem of minimizing the weighted sum of age of information (AoI) and total energy consumption of Internet of Things (IoT) devices is studied. In the considered model, each IoT device monitors a physical process that follows nonlinear dynamics. As the dynamics of the physical process vary over time, each device must find an optimal sampling frequency to sample the real-time dynamics of the physical system and send sampled information to a base station (BS). Due to limited wireless resources, the BS can only select a subset of devices to transmit their sampled information. Thus, edge devices must cooperatively sample their monitored dynamics based on the local observations and the BS must collect the sampled information from the devices immediately, hence avoiding the additional time and energy used for sampling and information transmission. To this end, it is necessary to jointly optimize the sampling policy of each device and the device selection scheme of the BS so as to accurately monitor the dynamics of the physical process using minimum energy. This problem is formulated as an optimization problem whose goal is to minimize the weighted sum of AoI cost and energy consumption. To solve this problem, we propose a novel distributed reinforcement learning (RL) approach for the sampling policy optimization. The proposed algorithm enables edge devices to cooperatively find the global optimal sampling policy using their own local observations. Given the sampling policy, the device selection scheme can be optimized thus minimizing the weighted sum of AoI and energy consumption of all devices. Simulations with real data of PM 2.5 pollution show that the proposed algorithm can reduce the sum of AoI by up to 17.8% and 33.9% and the total energy consumption by up to 13.2% and 35.1%, compared to a conventional deep Q network method and a uniform sampling policy.

AoI-Aware Resource Management for Smart Health Via Deep Reinforcement Learning

AoI-Aware Resource Scheduling for Industrial IoT with Deep Reinforcement Learning

A Deep Reinforcement Learning Approach for Improving Age of Information in Mission-Critical IoT

Distributed Reinforcement Learning for Age of Information Minimization in Real-Time IoT Systems

Optimal Status Updates for Minimizing Age of Correlated Information in IoT Networks with Energy Harvesting Sensors

AoI Minimization in Status Update Control with Energy Harvesting Sensors

Deep Reinforcement Learning Empowered Activity-Aware Dynamic Health Monitoring Systems

AoI-Oriented Resource Allocation for NOMA-Based Wireless Powered Cognitive Radio Networks Based on Multi-Agent Deep Reinforcement Learning

Adaptive Multi-Agent Deep Reinforcement Learning for Timely Healthcare Interventions

A context-aware sensing strategy with deep reinforcement learning for smart healthcare

Joint optimization of power control and time slot allocation for wireless body area networks via deep reinforcement learning

Reinforcement Learning (RL)-Based Energy Efficient Resource Allocation for Energy Harvesting-Powered Wireless Body Area Network

Deep Reinforcement Learning-Based Power Allocation for Minimizing Age of Information and Energy Consumption in Multi-Input Multi-Output and Non-Orthogonal Multiple Access Internet of Things Systems

Age-Based Scheduling for Mobile Edge Computing: A Deep Reinforcement Learning Approach

Deep Reinforcement Learning Based Multidimensional Resource Management for Energy Harvesting Cognitive NOMA Communications

Optimizing the Long-Term Average Reward for Continuing MDPs: A Technical Report

Dynamic Age Minimization With Real-Time Information Preprocessing for Edge-Assisted IoT Devices With Energy Harvesting

Multi-user Resource Control with Deep Reinforcement Learning in IoT Edge Computing

Hierarchical Deep Reinforcement Learning for Age-of-Information Minimization in IRS-aided and Wireless-powered Wireless Networks

An Effective Prediction of Resource Using Machine Learning in Edge Environments for the Smart Healthcare Industry

AoI and Energy Consumption Oriented Dynamic Status Updating in Caching Enabled IoT Networks