Abstract:In modern complex physical systems, advanced sensing technologies extend the sensor coverage but also increase the difficulties of improving system monitoring capabilities based on real-time data availability. Traditional model-based methods of sensor management are limited to specific systems/settings, which can be challenged when system knowledge is intractable. Fortunately, the large amount of data collected in real-time allows machine learning methods to be a complement. Especially, reinforcement learning-based control is recognized for its capability to dynamically interact with systems. However, the direct implementation of learning methods easily overfits and results in inaccurate physics modeling for sensor management. Although physical regularization is a popular direction to bridge the gap, learning-based sensor control still suffers from convergence failure under highly complex and uncertain scenarios. This paper develops physics-embedded and self-supervised reinforcement learning for sensor management using an intrinsic reward. Specifically, the intrinsic-motivated sensor management (IMSM) constructs the local surprise information from the physical latent features, which captures hidden states in observations, and thus intrinsically motivates the agent to speed-up exploration. We show that the designs can not only relieve the lack of consistency with underlying physics/physical dynamics, but also adapt the global objective of maximizing monitoring capabilities to local environment changes. We demonstrate its effectiveness by experiments on physical system sensor control. The proposed model is implemented for the sensor management of unmanned vehicles and sensor rescheduling in complex/settled power systems, with or without observability constraints. Numerical results show that our model provides consistently higher threat detection accuracy and better observability recovery, as compared to existing methods.

Intrinsic Motivation Driven Intuitive Physics Learning using Deep Reinforcement Learning with Intrinsic Reward Normalization

Intuitive physics learning in a deep-learning model inspired by developmental psychology

Learning Intuitive Physics and One-Shot Imitation Using State-Action-Prediction Self-Organizing Maps

Physics-Regulated Deep Reinforcement Learning: Invariant Embeddings

Learning to Play Video Games with Intuitive Physics Priors

3D-IntPhys: Towards More Generalized 3D-grounded Visual Intuitive Physics under Challenging Scenes

Intrinsic-Motivated Sensor Management: Exploring with Physical Surprise

Physics-Informed Model-Based Reinforcement Learning

DeepMimic: Example-Guided Deep Reinforcement Learning of Physics-Based Character Skills

Physics-Guided Hierarchical Reward Mechanism for Learning-Based Robotic Grasping

Developmental Reinforcement Learning through Sensorimotor Space Enlargement

Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivation

Emergence of Structured Behaviors from Curiosity-Based Intrinsic Motivation

Latent Intuitive Physics: Learning to Transfer Hidden Physics from A 3D Video

Curiosity-driven Intuitive Physics Learning

Combining physics and deep learning to learn continuous-time dynamics models

Strategy and Skill Learning for Physics-based Table Tennis Animation

Architecting and Visualizing Deep Reinforcement Learning Models

Intrinsic Motivation Exploration Via Self-Supervised Prediction in Reinforcement Learning

Deep Reinforcement Learning for 2D Physics-Based Object Manipulation in Clutter