Abstract:We tackle the challenge of robotic bin packing with irregular objects, such as groceries. Given the diverse physical attributes of these objects and the complex constraints governing their placement and manipulation, employing preprogrammed strategies becomes unfeasible. Our approach is to learn directly from expert demonstrations in order to extract implicit task knowledge and strategies to ensure safe object positioning, efficient use of space, and the generation of human-like behaviors that enhance human-robot trust. We rely on human demonstrations to learn a Markov chain for predicting the object packing sequence for a given set of items and then compare it with human performance. Our experimental results show that the model outperforms human performance by generating sequence predictions that humans classify as human-like more frequently than human-generated sequences. The human demonstrations were collected using our proposed VR platform, BoxED, which is a box packaging environment for simulating real-world objects and scenarios for fast and streamlined data collection with the purpose of teaching robots. We collected data from 43 participants packing a total of 263 boxes with supermarket-like objects, yielding 4644 object manipulations. Our VR platform can be easily adapted to new scenarios and objects, and is publicly available, alongside our dataset, at <a class="link-external link-https" href="https://github.com/andrejfsantos4/BoxED" rel="external noopener nofollow">this https URL</a>.

What problem does this paper attempt to address?

The problem that this paper attempts to solve is how robots can effectively pack irregular objects (such as groceries). Due to the diverse physical properties of these objects and the complex constraints on placement and manipulation, using pre - programmed strategies becomes infeasible. Therefore, the author proposes a method of learning from human demonstrations to extract implicit task knowledge and strategies, ensure the safe positioning of objects, the effective use of space, and generate human - like behaviors, thereby enhancing the trust between humans and machines. Specifically, the main contributions of the paper include: 1. **Model Trained on Datasets**: This model can predict the correct packing order and directly extract implicit task knowledge from human behaviors. The generated sequences not only ensure safe and efficient packing but are also rated by users as sequences closer to "human behaviors". 2. **Publicly Available VR Platform (BoxED)**: This platform is used for rapid and efficient data collection, can adapt to new packing scenarios and any object datasets, and collect 6 - degree - of - freedom grasping postures, object trajectories, packing orders, and object postures within the box. In addition, the first publicly available dataset of human demonstrations of packing groceries has been created. Through these contributions, the paper aims to solve the problems existing in current methods, such as search algorithms that only consider simple - shaped objects and methods that lack consideration of object characteristics (such as fragility). The paper experimentally demonstrates that the proposed model outperforms human performance in generating sequences that humans consider more like human behaviors.

Robotic Learning the Sequence of Packing Irregular Objects from Human Demonstrations

Learning the Sequence of Packing Irregular Objects from Human Demonstrations: Towards Autonomous Packing Robots

Towards reliable robot packing system based on deep reinforcement learning

Planning Irregular Object Packing Via Hierarchical Reinforcement Learning

Learning Few-Shot Object Placement with Intra-Category Transfer

Tight Robot Packing in the Real World: A Complete Manipulation Pipeline with Robust Primitives

Robot Online 3D Bin Packing Strategy Based on Deep Reinforcement Learning and 3D Vision

Learning Physically Realizable Skills for Online Packing of General 3D Shapes

Novel objects 3-D dense packing through robotic pushing

Learning Visual Robotic Control Efficiently with Contrastive Pre-training and Data Augmentation

A Method of Tight Placement for Robotic Dense Packing

RoboPack: Learning Tactile-Informed Dynamics Models for Dense Packing

Robotic Handling of Compliant Food Objects by Robust Learning from Demonstration

Robobarista: Object Part based Transfer of Manipulation Trajectories from Crowd-sourcing in 3D Pointclouds

Knowledge-Driven Robot Program Synthesis from Human VR Demonstrations

Robobarista: Learning to Manipulate Novel Objects via Deep Multimodal Embedding

BoxStacker: Deep Reinforcement Learning for 3D Bin Packing Problem in Virtual Environment of Logistics Systems

A Task-Learning Strategy for Robotic Assembly Tasks from Human Demonstrations

Fast Object Learning and Dual-arm Coordination for Cluttered Stowing, Picking, and Packing