Abstract:How to improve the ability of scene representation is a key issue in vision-oriented decision-making applications, and current approaches usually learn task-relevant state representations within visual reinforcement learning to address this problem. While prior work typically introduces one-step behavioral similarity metrics with elements (e.g., rewards and actions) to extract task-relevant state information from observations, they often ignore the inherent dynamics relationships among the elements that are essential for learning accurate representations, which further impedes the discrimination of short-term similar task/behavior information in long-term dynamics transitions. To alleviate this problem, we propose an intrinsic dynamics-driven representation learning method with sequence models in visual reinforcement learning, namely DSR. Concretely, DSR optimizes the parameterized encoder by the state-transition dynamics of the underlying system, which prompts the latent encoding information to satisfy the state-transition process and then the state space and the noise space can be distinguished. In the implementation and to further improve the representation ability of DSR on encoding similar tasks, sequential elements' frequency domain and multi-step prediction are adopted for sequentially modeling the inherent dynamics. Finally, experimental results show that DSR has achieved significant performance improvements in the visual Distracting DMControl control tasks, especially with an average of 78.9\% over the backbone baseline. Further results indicate that it also achieves the best performances in real-world autonomous driving applications on the CARLA simulator. Moreover, qualitative analysis results validate that our method possesses the superior ability to learn generalizable scene representations on visual tasks. The source code is available at <a class="link-external link-https" href="https://github.com/DMU-XMU/DSR" rel="external noopener nofollow">this https URL</a>.

Dynamics-Aware Context Representation for Domain Adaptation in Reinforcement Learning

Off-Dynamics Inverse Reinforcement Learning

Off-Dynamics Inverse Reinforcement Learning from Hetero-Domain

Dynamics-Adaptive Continual Reinforcement Learning Via Progressive Contextualization.

Unsupervised Domain Adaptation with Dynamics-Aware Rewards in Reinforcement Learning

Prototypical context-aware dynamics generalization for high-dimensional model-based reinforcement learning

Intrinsic Dynamics-Driven Generalizable Scene Representations for Vision-Oriented Decision-Making Applications

Reinforcement Learning with History-Dependent Dynamic Contexts

Deep Reinforcement Learning with Explicit Context Representation

Residual Learning and Context Encoding for Adaptive Offline-to-Online Reinforcement Learning

Transferable Reward Learning by Dynamics-Agnostic Discriminator Ensemble

Off-Dynamics Reinforcement Learning via Domain Adaptation and Reward Augmented Imitation

Domain Adaptation In Reinforcement Learning Via Latent Unified State Representation

A Relational Intervention Approach for Unsupervised Dynamics Generalization in Model-Based Reinforcement Learning

Episodic Reinforcement Learning with Expanded State-reward Space

Cross-Domain Policy Adaptation by Capturing Representation Mismatch

Masked and Inverse Dynamics Modeling for Data-Efficient Reinforcement Learning

Deep Attention Driven Reinforcement Learning (DAD-RL) for Autonomous Decision-Making in Dynamic Environment

Dynamics Generalization via Information Bottleneck in Deep Reinforcement Learning

Dynamic Context Removal: A General Training Strategy for Robust Models on Video Action Predictive Tasks

Dynamics-aware Embeddings