Task Planning for Object Rearrangement in Multi-room Environments

Karan Mirakhor,Sourav Ghosh,Dipanjan Das,Brojeshwar Bhowmick

2024-06-01

Abstract:Object rearrangement in a multi-room setup should produce a reasonable plan that reduces the agent's overall travel and the number of steps. Recent state-of-the-art methods fail to produce such plans because they rely on explicit exploration for discovering unseen objects due to partial observability and a heuristic planner to sequence the actions for rearrangement. This paper proposes a novel hierarchical task planner to efficiently plan a sequence of actions to discover unseen objects and rearrange misplaced objects within an untidy house to achieve a desired tidy state. The proposed method introduces several novel techniques, including (i) a method for discovering unseen objects using commonsense knowledge from large language models, (ii) a collision resolution and buffer prediction method based on Cross-Entropy Method to handle blocked goal and swap cases, (iii) a directed spatial graph-based state space for scalability, and (iv) deep reinforcement learning (RL) for producing an efficient planner. The planner interleaves the discovery of unseen objects and rearrangement to minimize the number of steps taken and overall traversal of the agent. The paper also presents new metrics and a benchmark dataset called MoPOR to evaluate the effectiveness of the rearrangement planning in a multi-room setting. The experimental results demonstrate that the proposed method effectively addresses the multi-room rearrangement problem.

Robotics

What problem does this paper attempt to address?

### The Problem Addressed by This Paper

This paper primarily addresses the task planning problem of object rearrangement in multi-room environments. Specifically:

1. **Efficient Planning under Partial Observability**:
- The paper proposes a novel hierarchical task planning method to tackle the object rearrangement problem under partially observable conditions in multi-room environments. This method aims to improve task efficiency by minimizing the agent's movement steps and total travel distance.

2. **Unseen Object Discovery**:
- A method for unseen object discovery based on large language models (LLMs) is proposed, utilizing common-sense knowledge to predict the possible rooms and container locations of unseen objects.

3. **Collision Resolution and Buffer Management**:
- A collision resolution strategy based on the Cross-Entropy Method is introduced to handle object collisions and situations where target positions are blocked or need to be swapped.

4. **Deep Reinforcement Learning Planner**:
- A Deep Reinforcement Learning (Deep RL) planner is used to generate effective action sequences for simultaneously searching for unseen objects and rearranging seen objects, thereby optimizing the agent's movement path and the number of steps.

5. **Benchmark Dataset**:
- To evaluate the effectiveness of the method, a new benchmark dataset, MoPOR, is introduced for testing task planners in multi-room environments, overcoming the limitations of existing datasets.

In summary, the main goal of this paper is to achieve efficient object rearrangement tasks in multi-room environments under partially observable conditions through effective planning strategies, while reducing the agent's travel distance and number of steps.