Abstract:Learning from human demonstrations is an emerging trend for designing intelligent robotic systems. However, previous methods typically regard videos as instructions, simply dividing them into action sequences for robotic repetition, which poses obstacles to generalization to diverse tasks or object instances. In this paper, we propose a different perspective, considering human demonstration videos not as mere instructions, but as a source of knowledge for robots. Motivated by this perspective and the remarkable comprehension and generalization capabilities exhibited by large language models (LLMs), we propose DigKnow, a method that DIstills Generalizable KNOWledge with a hierarchical structure. Specifically, DigKnow begins by converting human demonstration video frames into observation knowledge. This knowledge is then subjected to analysis to extract human action knowledge and further distilled into pattern knowledge compassing task and object instances, resulting in the acquisition of generalizable knowledge with a hierarchical structure. In settings with different tasks or object instances, DigKnow retrieves relevant knowledge for the current task and object instances. Subsequently, the LLM-based planner conducts planning based on the retrieved knowledge, and the policy executes actions in line with the plan to achieve the designated task. Utilizing the retrieved knowledge, we validate and rectify planning and execution outcomes, resulting in a substantial enhancement of the success rate. Experimental results across a range of tasks and scenes demonstrate the effectiveness of this approach in facilitating real-world robots to accomplish tasks with the knowledge derived from human demonstrations.

Shaping in Reinforcement Learning by Knowledge Transferred from Human-Demonstrations of a Simple Similar Task.

Shaping in Reinforcement Learning Via Knowledge Transferred from Human-Demonstrations

Transferring knowledge from human-demonstration trajectories to reinforcement learning

Shaping Reward Learning Approach from Passive Samples

Efficient Reinforcement Learning from Demonstration Via Bayesian Network-Based Knowledge Extraction

Reward Shaping via Meta-Learning

Learning Shaping Strategies in Human-in-the-loop Interactive Reinforcement Learning.

Learning to Utilize Shaping Rewards: A New Approach of Reward Shaping

DGTRL: Deep graph transfer reinforcement learning method based on fusion of knowledge and data

Human Demonstrations are Generalizable Knowledge for Robots

From Reward Shaping to Q-Shaping: Achieving Unbiased Learning with LLM-Guided Knowledge

Mutual Information Based Knowledge Transfer Under State-Action Dimension Mismatch

Reusing Source Task Knowledge Via Transfer Approximator in Reinforcement Transfer Learning

Hierarchical Reinforcement Learning from Demonstration via Reachability-Based Reward Shaping

Knowledge Sharing and Transfer via Centralized Reward Agent for Multi-Task Reinforcement Learning

KnowRU: Knowledge Reusing via Knowledge Distillation in Multi-agent Reinforcement Learning

KnowRU: Knowledge Reuse Via Knowledge Distillation in Multi-Agent Reinforcement Learning

A new Potential-Based Reward Shaping for Reinforcement Learning Agent

KnowSR: Knowledge Sharing among Homogeneous Agents in Multi-agent Reinforcement Learning

Reinforcement Learning Transfer Based on Subgoal Discovery and Subtask Similarity

Shaping Progressive Net of Reinforcement Learning for Policy Transfer with Human Evaluative Feedback