Abstract:In the early stages of human life, babies develop their skills by exploring different scenarios motivated by their inherent satisfaction rather than by extrinsic rewards from the environment. This behavior, referred to as intrinsic motivation, has emerged as one solution to address the exploration challenge derived from reinforcement learning environments with sparse rewards. Diverse exploration approaches have been proposed to accelerate the learning process over single- and multi-agent problems with homogeneous agents. However, scarce studies have elaborated on collaborative learning frameworks between heterogeneous agents deployed into the same environment, but interacting with different instances of the latter without any prior knowledge. Beyond the heterogeneity, each agent's characteristics grant access only to a subset of the full state space, which may hide different exploration strategies and optimal solutions. In this work we combine ideas from intrinsic motivation and transfer learning. Specifically, we focus on sharing parameters in actor-critic model architectures and on combining information obtained through intrinsic motivation with the aim of having a more efficient exploration and faster learning. We test our strategies through experiments performed over a modified ViZDooM's My Way Home scenario, which is more challenging than its original version and allows evaluating the heterogeneity between agents. Our results reveal different ways in which a collaborative framework with little additional computational cost can outperform an independent learning process without knowledge sharing. Additionally, we depict the need for modulating correctly the importance between the extrinsic and intrinsic rewards to avoid undesired agent behaviors.

Improving interactive reinforcement learning: What makes a good teacher?

Social Interaction for Efficient Agent Learning from Human Reward.

Policy Learning with Human Reinforcement

Teachable Reinforcement Learning via Advice Distillation

Interactive Learning from Policy-Dependent Human Feedback

Learning from Human Reward Benefits from Socio-Competitive Feedback

Optimal Exploration Algorithm of Multi-Agent Reinforcement Learning Methods (Student Abstract)

AC-Teach: A Bayesian Actor-Critic Method for Policy Learning with an Ensemble of Suboptimal Teachers

Improving Policy Generalization for Teacher-Student Reinforcement Learning.

Learning by Reusing Previous Advice in Teacher-Student Paradigm.

Learning Shaping Strategies in Human-in-the-loop Interactive Reinforcement Learning.

Collaborative training of heterogeneous reinforcement learning agents in environments with sparse rewards: what and when to share?

Reinforcement Learning for Improving Agent Design

Integrating human learning and reinforcement learning: A novel approach to agent training

Influencing Reinforcement Learning through Natural Language Guidance

Enhancing End-to-End Multi-Task Dialogue Systems: A Study on Intrinsic Motivation Reinforcement Learning Algorithms for Improved Training and Adaptability

Best Response Shaping

Don't Forget Your Teacher: A Corrective Reinforcement Learning Framework

Adaptively Shaping Reinforcement Learning Agents Via Human Reward

Reinforcement Teaching

Agent-Aware Training for Agent-Agnostic Action Advising in Deep Reinforcement Learning