Abstract:Purpose Autonomous robots must be able to understand long-term manipulation tasks described by humans and perform task analysis and planning based on the current environment in a variety of scenes, such as daily manipulation and industrial assembly. However, both classical task and motion planning algorithms and single data-driven learning planning methods have limitations in practicability, generalization and interpretability. The purpose of this work is to overcome the limitations of the above methods and achieve generalized and explicable long-term robot manipulation task planning. Design/methodology/approach The authors propose a planning method for long-term manipulation tasks that combines the advantages of existing methods and the prior cognition brought by the knowledge graph. This method integrates visual semantic understanding based on scene graph generation, regression planning based on deep learning and multi-level representation and updating based on a knowledge base. Findings The authors evaluated the capability of this method in a kitchen cooking task and tabletop arrangement task in simulation and real-world environments. Experimental results show that the proposed method has a significantly improved success rate compared with the baselines and has excellent generalization performance for new tasks. Originality/value The authors demonstrate that their method is scalable to long-term manipulation tasks with varying complexity and visibility. This advantage allows their method to perform better in new manipulation tasks. The planning method proposed in this work is meaningful for the present robot manipulation task and can be intuitive for similar high-level robot planning.

Task Planning with a Weighted Functional Object-Oriented Network

Whole-Body Inverse Kinematics and Operation-Oriented Motion Planning for Robot Mobile Manipulation

Approach To Cooperative Assembly Task Planning For Multiple Manipulators

Human Robot Collaboration for Enhancing Work Activities.

Robot Task Planning Based on Large Language Model Representing Knowledge with Directed Graph Structures

Task tree retrieval from FOON using search algorithms

Long-term Robot Manipulation Task Planning with Scene Graph and Semantic Knowledge

Task Allocation for Multi-Robot Task and Motion Planning: a case for Object Picking in Cluttered Workspaces

An Optimal Task Planning and Agent-aware Allocation Algorithm in Collaborative Tasks Combining with PDDL and POPF

Toward coordinated planning and hierarchical optimization control for highly redundant mobile manipulator

A Framework to Co-Optimize Robot Exploration and Task Planning in Unknown Environments

Manipulation planning under changing external forces

Object-Centric Kinodynamic Planning for Nonprehensile Robot Rearrangement Manipulation

Human-in-the-loop Robotic Manipulation Planning for Collaborative Assembly

An Online Task-Planning Framework Using Mixed Integer Programming for Multiple Cooking Tasks Using a Dual-Arm Robot

Object-Oriented Option Framework for Robotics Manipulation in Clutter.

Robot Task Planning and Situation Handling in Open Worlds

A Task Allocation Framework for Human Multi-Robot Collaborative Settings

Design Optimizer for Planar Soft-Growing Robot Manipulators

An Integrated Dynamic Method for Allocating Roles and Planning Tasks for Mixed Human-Robot Teams