Abstract:Collective human knowledge has clearly benefited from the fact that innovations by individuals are taught to others through communication. Similar to human social groups, agents in distributed learning systems would likely benefit from communication to share knowledge and teach skills. The problem of teaching to improve agent learning has been investigated by prior works, but these approaches make assumptions that prevent application of teaching to general multiagent problems, or require domain expertise for problems they can apply to. This learning to teach problem has inherent complexities related to measuring long-term impacts of teaching that compound the standard multiagent coordination challenges. In contrast to existing works, this paper presents the first general framework and algorithm for intelligent agents to learn to teach in a multiagent environment. Our algorithm, Learning to Coordinate and Teach Reinforcement (LeCTR), addresses peer-to-peer teaching in cooperative multiagent reinforcement learning. Each agent in our approach learns both when and what to advise, then uses the received advice to improve local learning. Importantly, these roles are not fixed; these agents learn to assume the role of student and/or teacher at the appropriate moments, requesting and providing advice in order to improve teamwide performance and learning. Empirical comparisons against state-of-the-art teaching methods show that our teaching agents not only learn significantly faster, but also learn to coordinate in tasks where existing methods fail.

Teachable Reinforcement Learning via Advice Distillation

Learning by Reusing Previous Advice in Teacher-Student Paradigm.

Learning to Teach Reinforcement Learning Agents

Towards Teachable Autotelic Agents

Self-Attention Guided Advice Distillation in Multi-Agent Deep Reinforcement Learning

Learning on a Budget via Teacher Imitation

Action Advising with Advice Imitation in Deep Reinforcement Learning

PEBBLE: Feedback-Efficient Interactive Reinforcement Learning via Relabeling Experience and Unsupervised Pre-training

Improving interactive reinforcement learning: What makes a good teacher?

Learning to Teach in Cooperative Multiagent Reinforcement Learning

Using Machine Teaching to Investigate Human Assumptions when Teaching Reinforcement Learners

Teacher-Student Framework: a Reinforcement Learning Approach

Influencing Reinforcement Learning through Natural Language Guidance

Agent-Aware Training for Agent-Agnostic Action Advising in Deep Reinforcement Learning

AI2T: Building Trustable AI Tutors by Interactively Teaching a Self-Aware Learning Agent

Teaching to Learn: Sequential Teaching of Agents with Inner States

Rethinking Supervised Learning and Reinforcement Learning in Task-Oriented Dialogue Systems

Leveraging Human Guidance for Deep Reinforcement Learning Tasks

Interactively Teaching an Inverse Reinforcement Learner with Limited Feedback

Teaching a Robot Tasks of Arbitrary Complexity Via Human Feedback

AC-Teach: A Bayesian Actor-Critic Method for Policy Learning with an Ensemble of Suboptimal Teachers