Abstract:Foundation models (FMs), large deep learning models pre-trained on vast, unlabeled datasets, exhibit powerful capabilities in understanding complex patterns and generating sophisticated outputs. However, they often struggle to adapt to specific tasks. Reinforcement learning (RL), which allows agents to learn through interaction and feedback, offers a compelling solution. Integrating RL with FMs enables these models to achieve desired outcomes and excel at particular tasks. Additionally, RL can be enhanced by leveraging the reasoning and generalization capabilities of FMs. This synergy is revolutionizing various fields, including robotics. FMs, rich in knowledge and generalization, provide robots with valuable information, while RL facilitates learning and adaptation through real-world interactions. This survey paper comprehensively explores this exciting intersection, examining how these paradigms can be integrated to advance robotic intelligence. We analyze the use of foundation models as action planners, the development of robotics-specific foundation models, and the mutual benefits of combining FMs with RL. Furthermore, we present a taxonomy of integration approaches, including large language models, vision-language models, diffusion models, and transformer-based RL models. We also explore how RL can utilize world representations learned from FMs to enhance robotic task execution. Our survey aims to synthesize current research and highlight key challenges in robotic reasoning and control, particularly in the context of integrating FMs and RL--two rapidly evolving technologies. By doing so, we seek to spark future research and emphasize critical areas that require further investigation to enhance robotics. We provide an updated collection of papers based on our taxonomy, accessible on our open-source project website at: <a class="link-external link-https" href="https://github.com/clmoro/Robotics-RL-FMs-Integration" rel="external noopener nofollow">this https URL</a>.

Special issue on real-world robot applications of the foundation models

Real-World Robot Applications of Foundation Models: A Review

Foundation models in robotics: Applications, challenges, and the future

Introduction to the Focused Section on New Trends in Modelling and Simulation for Intelligent Robotics

Robot Learning in the Era of Foundation Models: A Survey

Toward General-Purpose Robots via Foundation Models: A Survey and Meta-Analysis

A Survey on Robotics with Foundation Models: toward Embodied AI

Foundation Models for Decision Making: Problems, Methods, and Opportunities

Foundation Models for Autonomous Robots in Unstructured Environments

Integrating Reinforcement Learning with Foundation Models for Autonomous Robotics: Methods and Perspectives

Transferring Foundation Models for Generalizable Robotic Manipulation

Foundation Models for Remote Sensing and Earth Observation: A Survey

What Foundation Models can Bring for Robot Learning in Manipulation : A Survey

Pave the Way to Grasp Anything: Transferring Foundation Models for Universal Pick-Place Robots

Prospective Role of Foundation Models in Advancing Autonomous Vehicles

Foundation Models Meet Visualizations: Challenges and Opportunities

Sora for Foundation Robots with Parallel Intelligence: Three World Models, Three Robotic Systems

A Taxonomy of Foundation Model based Systems through the Lens of Software Architecture