Abstract:Imitation learning is a widely-used paradigm for decision making that learns from expert demonstrations. Existing imitation algorithms often require multiple interactions between the agent and the environment from which the demonstration is obtained. The acquisition of expert demonstrations in simulator usually requires specialized knowledge. In addition, real-world interactions are limited due to security or cost concerns. Therefore, the direct application of existing imitation learning algorithms in either real world or simulator is not an ideal strategy. In this paper, we propose a cross-domain Inverse Reinforcement Learning training paradigm that learns a reward function from hetero-domain expert’s demonstration, while the interaction with the environment that obtains demonstrations should be limited. In order to solve the distribution shift under such paradigm, we propose a transfer learning method called off-dynamics Inverse Reinforcement Learning. The intuition behind off-dynamics Inverse Reinforcement Learning is that the goal of reward function learning is not only to imitate experts, but also to promote action adaptation to the dynamic difference between two hetero-domain. Specifically, a widely-used Inverse Reinforcement Learning framework was adopted, and its discriminator for identifying agent-generated trajectories was modified with quantified dynamic differences. The training process of the discriminator yields the transferable reward function suitable for the target dynamics, which is guaranteed by our theoretical derivation. Off-dynamics Inverse Reinforcement Learning assigns higher rewards to demonstration trajectories that do not exploit discrepancies between the two domains. Our method demonstrates its effectiveness and scalability to high-dimensional tasks through extensive experiments on continuous control tasks. Our code is available on the project website: https://github.com/yachenkang/ODIRL.

Reinforcement Learning for SAR Target Orientation Inference with the Differentiable SAR Renderer

Reinforcement Learning for SAR View Angle Inversion with Differentiable SAR Renderer

Differentiable SAR Renderer Embedded Reinforcement Learning for View Angles Inversion in SAR Images

Off-Dynamics Inverse Reinforcement Learning from Hetero-Domain

Off-Dynamics Inverse Reinforcement Learning

SAR Image Generation by Integrating Differentiable SAR Renderer with Neural Networks

Differentiable SAR Renderer and SAR Target Reconstruction

Differentiable SAR Renderer and Image-Based Target Reconstruction

A Reinforcement Learning Framework for Scattering Feature Extraction and SAR Image Interpretation

Differentiable Rendering for Synthetic Aperture Radar Imagery

Sequential Action-Induced Invariant Representation for Reinforcement Learning

Learning Surface Scattering Parameters From SAR Images Using Differentiable Ray Tracing

Distance-rank Aware Sequential Reward Learning for Inverse Reinforcement Learning with Sub-optimal Demonstrations

Domain Adaptive State Representation Alignment for Reinforcement Learning

Extension of Differentiable SAR Renderer for Ground Target Reconstruction from Multiview Images and Shadows

SAR-NeRF: Neural Radiance Fields for Synthetic Aperture Radar Multi-View Representation

Rotation Awareness Based Self-Supervised Learning for SAR Target Recognition With Limited Training Samples

Learning Controllable Elements Oriented Representations for Reinforcement Learning

A Resource Scheduling Algorithm for Multi-Target 3D Imaging in Radar Network Based on Deep Reinforcement Learning

Reinforcement Learning for Orientation Estimation Using Inertial Sensors with Performance Guarantee