Abstract:Wireless networked control system (WNCS) connecting sensors, controllers, and actuators via wireless communications is a key enabling technology for highly scalable and low-cost deployment of control systems in the Industry 4.0 era. Despite the tight interaction of control and communications in WNCSs, most existing works adopt separative design approaches. This is mainly because the co-design of control-communication policies requires large and hybrid state and action spaces, making the optimal problem mathematically intractable and difficult to be solved effectively by classic algorithms. In this paper, we systematically investigate deep learning (DL)-based estimator-control-scheduler co-design for a model-unknown nonlinear WNCS over wireless fading channels. In particular, we propose a co-design framework with the awareness of the sensor's age-of-information (AoI) states and dynamic channel states. We propose a novel deep reinforcement learning (DRL)-based algorithm for controller and scheduler optimization utilizing both model-free and model-based data. An AoI-based importance sampling algorithm that takes into account the data accuracy is proposed for enhancing learning efficiency. We also develop novel schemes for enhancing the stability of joint training. Extensive experiments demonstrate that the proposed joint training algorithm can effectively solve the estimation-control-scheduling co-design problem in various scenarios and provide significant performance gain compared to separative design and some benchmark policies.

Optimal Resource Allocation in Wireless Control Systems via Deep Policy Gradient

Deep-Learning-Based Wireless Resource Allocation With Application to Vehicular Networks

Deep Learning based Wireless Resource Allocation with Application to Vehicular Networks

Power Allocation for Full-Duplex Communication Systems Based on Deep Deterministic Policy Gradient

Deep Reinforcement Learning-based Power Control and Bandwidth Allocation Policy for Weighted Cost Minimization in Wireless Networks

DDPG with Transfer Learning and Meta Learning Framework for Resource Allocation in Underlay Cognitive Radio Network

Optimization Theory Based Deep Reinforcement Learning for Resource Allocation in Ultra-Reliable Wireless Networked Control Systems

Autonomous Management of Energy-Harvesting IoT Nodes Using Deep Reinforcement Learning

Deep Learning for Wireless Networked Systems: a joint Estimation-Control-Scheduling Approach

Deep Reinforcement Learning for Wireless Scheduling in Distributed Networked Control

Deep Deterministic Policy Gradient for Relay Selection and Power Allocation in Cooperative Communication Network

Optimal Path Planning for Wireless Power Transfer Robot Using Area Division Deep Reinforcement Learning

Deep Deterministic Policy Gradient-Based Resource Allocation Considering Network Slicing and Device-to-Device Communication in Mobile Networks

Deep Reinforcement Learning Optimal Transmission Algorithm for Cognitive Internet of Things with RF Energy Harvesting

Learning Deterministic Policy with Target for Power Control in Wireless Networks

Deep Reinforcement Learning for Radio Resource Allocation in NOMA-based Remote State Estimation

DRL-based Resource Allocation in Remote State Estimation

Reinforcement Learning Based Power Control for Reliable Mission-Critical Wireless Transmission

Optimal Coordination of Distributed Energy Resources Using Deep Deterministic Policy Gradient

Energy‐efficient resource allocation over wireless communication systems through deep reinforcement learning