Abstract:In modern complex physical systems, advanced sensing technologies extend the sensor coverage but also increase the difficulties of improving system monitoring capabilities based on real-time data availability. Traditional model-based methods of sensor management are limited to specific systems/settings, which can be challenged when system knowledge is intractable. Fortunately, the large amount of data collected in real-time allows machine learning methods to be a complement. Especially, reinforcement learning-based control is recognized for its capability to dynamically interact with systems. However, the direct implementation of learning methods easily overfits and results in inaccurate physics modeling for sensor management. Although physical regularization is a popular direction to bridge the gap, learning-based sensor control still suffers from convergence failure under highly complex and uncertain scenarios. This paper develops physics-embedded and self-supervised reinforcement learning for sensor management using an intrinsic reward. Specifically, the intrinsic-motivated sensor management (IMSM) constructs the local surprise information from the physical latent features, which captures hidden states in observations, and thus intrinsically motivates the agent to speed-up exploration. We show that the designs can not only relieve the lack of consistency with underlying physics/physical dynamics, but also adapt the global objective of maximizing monitoring capabilities to local environment changes. We demonstrate its effectiveness by experiments on physical system sensor control. The proposed model is implemented for the sensor management of unmanned vehicles and sensor rescheduling in complex/settled power systems, with or without observability constraints. Numerical results show that our model provides consistently higher threat detection accuracy and better observability recovery, as compared to existing methods.

Optimal Policies Search for Sensor Management

Stochastic Steepest-Descent Optimization Of Multiple-Objective Mobile Sensor Coverage

Multi-objective Sensor Management Method Based on Twin Delayed Deep Deterministic policy gradient algorithm

Decomposed POMDP Optimization-Based Sensor Management for Multi-Target Tracking in Passive Multi-Sensor Systems

An efficient multi-objective optimization approach for sensor management via multi-Bernoulli filtering

Airborne Self-adaptive Multi-sensor Management.

Optimizing Sensor Redundancy in Sequential Decision-Making Problems

Intrinsic-Motivated Sensor Management: Exploring with Physical Surprise

Policy Search for the Optimal Control of Markov Decision Processes: A Novel Particle-Based Iterative Scheme

Optimal Sensor Positioning (OSP); A Probability Perspective Study

Deep Optimal Sensor Placement for Black Box Stochastic Simulations

Optimizing pre-scheduled, intermittently-observed MDPs

Adaptive Policies for Perimeter Surveillance Problems

Multisensor Management Algorithm for Airborne Sensors Using Frank-Wolfe Method

Sensor Activation Policy Optimization for Opacity Enforcement Based on Reinforcement Learning

Robust Action Selection in Partially Observable Markov Decision Processes with Model Uncertainty

On Solving Optimal Policies for Finite-Stage Event-Based Optimization

Utility Maximizing Sequential Sensing Over a Finite Horizon

Entropic Risk Measure in Policy Search

Deep reinforcement learning driven inspection and maintenance planning under incomplete information and constraints

Optimal Sensor Placement Design for Profile Estimation of Distributed Parameter Systems