Abstract:Imitation learning is a widely-used paradigm for decision making that learns from expert demonstrations. Existing imitation algorithms often require multiple interactions between the agent and the environment from which the demonstration is obtained. The acquisition of expert demonstrations in simulator usually requires specialized knowledge. In addition, real-world interactions are limited due to security or cost concerns. Therefore, the direct application of existing imitation learning algorithms in either real world or simulator is not an ideal strategy. In this paper, we propose a cross-domain Inverse Reinforcement Learning training paradigm that learns a reward function from hetero-domain expert’s demonstration, while the interaction with the environment that obtains demonstrations should be limited. In order to solve the distribution shift under such paradigm, we propose a transfer learning method called off-dynamics Inverse Reinforcement Learning. The intuition behind off-dynamics Inverse Reinforcement Learning is that the goal of reward function learning is not only to imitate experts, but also to promote action adaptation to the dynamic difference between two hetero-domain. Specifically, a widely-used Inverse Reinforcement Learning framework was adopted, and its discriminator for identifying agent-generated trajectories was modified with quantified dynamic differences. The training process of the discriminator yields the transferable reward function suitable for the target dynamics, which is guaranteed by our theoretical derivation. Off-dynamics Inverse Reinforcement Learning assigns higher rewards to demonstration trajectories that do not exploit discrepancies between the two domains. Our method demonstrates its effectiveness and scalability to high-dimensional tasks through extensive experiments on continuous control tasks. Our code is available on the project website: https://github.com/yachenkang/ODIRL.

OTF Gym: A Set of Reinforcement Learning Environment of Layered Optical Thin Film Inverse Design

Off-Dynamics Inverse Reinforcement Learning

A Reinforcement learning method for Optical Thin-Film Design

Multilayer Optical Thin Film Design with Deep Q Learning

OptoGPT: A Foundation Model for Inverse Design in Optical Multilayer Thin Film Structures

TMM-Fast: A Transfer Matrix Computation Package for Multilayer Thin-Film Optimization

Automated Optical Multi-layer Design via Deep Reinforcement Learning

Automated multi-layer optical design via deep reinforcement learning

A new multilayer optical film optimal method based on deep q-learning.

Deep Neural Network-Based Inverse Design of a Novel Reconfigurable Multi-Tap Optical Filter

invrs-gym: a toolkit for nanophotonic inverse design research

Thin-film Neural Networks for Optical Inverse Problem

A manufacture-friendly design framework for optical neural networks

Data-driven design of thin-film optical systems using deep active learning

L2O-ILT: Learning to Optimize Inverse Lithography Techniques

Inverse Design of Composite Metal Oxide Optical Materials based on Deep Transfer Learning

Efficient inverse design of optical multilayer nano-thin films using neural network principles: backpropagation and gradient descent

Progressive Multi-Modality Learning for Inverse Protein Folding

OffWorld Gym: open-access physical robotics environment for real-world reinforcement learning benchmark and research

Structural Design Through Reinforcement Learning

Curriculum Learning for ab initio Deep Learned Refractive Optics