A Comprehensive Survey of Cross-Domain Policy Transfer for Embodied Agents

Haoyi Niu,Jianming Hu,Guyue Zhou,Xianyuan Zhan

2024-08-27

Abstract:The burgeoning fields of robot learning and embodied AI have triggered an increasing demand for large quantities of data. However, collecting sufficient unbiased data from the target domain remains a challenge due to costly data collection processes and stringent safety requirements. Consequently, researchers often resort to data from easily accessible source domains, such as simulation and laboratory environments, for cost-effective data acquisition and rapid model iteration. Nevertheless, the environments and embodiments of these source domains can be quite different from their target domain counterparts, underscoring the need for effective cross-domain policy transfer approaches. In this paper, we conduct a systematic review of existing cross-domain policy transfer methods. Through a nuanced categorization of domain gaps, we encapsulate the overarching insights and design considerations of each problem setting. We also provide a high-level discussion about the key methodologies used in cross-domain policy transfer problems. Lastly, we summarize the open challenges that lie beyond the capabilities of current paradigms and discuss potential future directions in this field.

Robotics,Artificial Intelligence,Machine Learning

What problem does this paper attempt to address?

This paper aims to solve the problem that in the fields of robot learning and embodied AI, due to the high cost of data collection in the target domain (such as the real world or task environment) and strict safety requirements, it is difficult to obtain sufficient unbiased data. To address this challenge, researchers usually use data from easily accessible source domains (such as simulation or laboratory environments) for cost - effective data collection and rapid model iteration. However, the environments and embodied forms of these source domains may be significantly different from those of the target domain, which emphasizes the importance of effective cross - domain policy transfer methods. This article systematically reviews the existing cross - domain policy transfer methods, makes a detailed classification of different types of domain gaps, and discusses the overall insights and design considerations for each problem setting. In addition, the article also discusses the key methodologies for cross - domain policy transfer problems and summarizes the open challenges that the current paradigms cannot solve and future research directions.

A Comprehensive Survey of Cross-Domain Policy Transfer for Embodied Agents

Cross Domain Policy Transfer with Effect Cycle-Consistency

Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Survey

Cross-Domain Policy Transfer by Representation Alignment via Multi-Domain Behavioral Cloning

Policy Transfer across Visual and Dynamics Domain Gaps via Iterative Grounding

Modular Deep Q Networks for Sim-to-real Transfer of Visuo-motor Policies

Cross-Modal Domain Adaptation for Reinforcement Learning

Knowledge Transfer for Cross-Domain Reinforcement Learning: A Systematic Review

Policy Transfer with Strategy Optimization

Cross-Embodiment Robot Manipulation Skill Transfer using Latent Space Alignment

A Framework for Few-Shot Policy Transfer through Observation Mapping and Behavior Cloning

xTED: Cross-Domain Adaptation via Diffusion-Based Trajectory Editing

Policy Stitching: Learning Transferable Robot Policies

Know Thyself: Transferable Visual Control Policies Through Robot-Awareness

Pushing the Limits of Cross-Embodiment Learning for Manipulation and Navigation

Scaling Cross-Embodied Learning: One Policy for Manipulation, Navigation, Locomotion and Aviation

Cross-domain policy adaptation with dynamics alignment

Efficient Data Collection for Robotic Manipulation via Compositional Generalization

TRANSIC: Sim-to-Real Policy Transfer by Learning from Online Correction

Cross-Domain Policy Adaptation via Value-Guided Data Filtering

Transfer learning in robotics: An upcoming breakthrough? A review of promises and challenges