Abstract:In this paper, the problem of minimizing the weighted sum of age of information (AoI) and total energy consumption of Internet of Things (IoT) devices is studied. In the considered model, each IoT device monitors a physical process that follows nonlinear dynamics. As the dynamics of the physical process vary over time, each device must find an optimal sampling frequency to sample the real-time dynamics of the physical system and send sampled information to a base station (BS). Due to limited wireless resources, the BS can only select a subset of devices to transmit their sampled information. Thus, edge devices must cooperatively sample their monitored dynamics based on the local observations and the BS must collect the sampled information from the devices immediately, hence avoiding the additional time and energy used for sampling and information transmission. To this end, it is necessary to jointly optimize the sampling policy of each device and the device selection scheme of the BS so as to accurately monitor the dynamics of the physical process using minimum energy. This problem is formulated as an optimization problem whose goal is to minimize the weighted sum of AoI cost and energy consumption. To solve this problem, we propose a novel distributed reinforcement learning (RL) approach for the sampling policy optimization. The proposed algorithm enables edge devices to cooperatively find the global optimal sampling policy using their own local observations. Given the sampling policy, the device selection scheme can be optimized thus minimizing the weighted sum of AoI and energy consumption of all devices. Simulations with real data of PM 2.5 pollution show that the proposed algorithm can reduce the sum of AoI by up to 17.8% and 33.9% and the total energy consumption by up to 13.2% and 35.1%, compared to a conventional deep Q network method and a uniform sampling policy.

Lifelong Learning for Minimizing Age of Information in Internet of Things Networks

UAV-Aided Lifelong Learning for AoI and Energy Optimization in Non-Stationary IoT Networks

Reinforcement Learning for Minimizing Age of Information under Realistic Physical Dynamics

Reinforcement Learning for Minimizing Age of Information in Real-time Internet of Things Systems with Realistic Physical Dynamics.

Multi-UAV Path Learning for Age and Power Optimization in IoT With UAV Battery Recharge

Deep Reinforcement Learning-Based Solution for Minimizing the Alterable Urgency of Information in UAV- Enabled IIoT System

Age Minimization in Massive IoT via UAV Swarm: A Multi-agent Reinforcement Learning Approach

Reinforcement Learning for Optimizing Age of Information and Energy Consumption in Misson-critical IoT

Age of Information Minimization for UAV-assisted Internet of Things Networks: A Safe Actor-Critic with Policy Distillation Approach

Age of Information Optimization with Heterogeneous UAVs Based on Deep Reinforcement Learning

Learning-Based Data Gathering for Information Freshness in UAV-Assisted IoT Networks

Distributed Reinforcement Learning for Age of Information Minimization in Real-Time IoT Systems

Energy-aware Path Planning for Obtaining Fresh Updates in UAV-IoT MEC systems

Intelligent Scheduling of UAVs and Sensors for Information Age Minimization at Wireless Powered Internet of Things

Distributed Reinforcement Learning for Optimizing Age of Information and Energy Consumption in Wireless Powered IoT Systems.

Deep Reinforcement Learning for IoT Networks: Age of Information and Energy Cost Tradeoff

Deep Reinforcement Learning for Efficient Data Collection in UAV-Aided Internet of Things

UAV-RIS Assisted AoI Minimization in IoT Networks: A Reinforcement Learning Approach

A Learning-Based Trajectory Planning of Multiple UAVs for AoI Minimization in IoT Networks

Minimizing Age of Information for Hybrid UAV-RIS-Assisted Vehicular Networks

A Deep Reinforcement Learning Approach for Improving Age of Information in Mission-Critical IoT