Abstract:Embodied AI, where agents accomplish specific tasks through interaction with their surrounding environment, is attracting attention in the community. As a more comprehensive and practical embodied task, visual room rearrangement aims to rearrange the initially misplaced objects in a room to the target configuration. In this task, the misplaced targets are mostly small objects that are difficult to locate, and there is no indication of which object needs to be rearranged in sensory input. Direct search for small and unknown targets poses a challenge. Previous works tend to directly incorporate sensory input into a reward-based framework. These methods do not learn the relationships between objects and infer which objects need to be rearranged, resulting in low performance. Understanding the scene structure and reasoning task goals can enable agents to rearrange targets more efficiently. In this paper, we propose a receptacle-target relation graph, which considers the spatial relationship between target objects and the receptacle objects appearing around them, enabling agents to understand the environment. Furthermore, we also propose a target reasoning network to explicitly predict the target object. Finally, our goal-aware policy can plan a series of actions based on their understanding of the task goal. Extensive compmethodarative and ablation experiments were conducted on the AI2Thor platform, demonstrating the effectiveness of our method.

Rearrangement: A challenge for embodied ai

Visual Room Rearrangement

Learning Object Relation Graph and Goal-Aware Policy for Visual Room Rearrangement

SplatR : Experience Goal Visual Rearrangement with 3D Gaussian Splatting and Dense Feature Matching

Stimulating Imagination: Towards General-purpose Object Rearrangement

Phase Discriminated Multi-Policy for Visual Room Rearrangement.

Task Planning for Object Rearrangement in Multi-room Environments

MANER: Multi-Agent Neural Rearrangement Planning of Objects in Cluttered Environments

Neural Constraint Satisfaction: Hierarchical Abstraction for Combinatorial Generalization in Object Rearrangement

Rearrange Indoor Scenes for Human-Robot Co-Activity

Adaptive coordination in social embodied rearrangement

ConSOR: A Context-Aware Semantic Object Rearrangement Framework for Partially Arranged Scenes

Object Rearrangement with Nested Nonprehensile Manipulation Actions

SG-Bot: Object Rearrangement via Coarse-to-Fine Robotic Imagination on Scene Graphs

Embodied Referring Expression for Manipulation Question Answering in Interactive Environment

Shelving, Stacking, Hanging: Relational Pose Diffusion for Multi-modal Rearrangement

FAVOR: Full-Body AR-Driven Virtual Object Rearrangement Guided by Instruction Text

Object Rearrangement Using Learned Implicit Collision Functions

PEARL: Parallelized Expert-Assisted Reinforcement Learning for Scene Rearrangement Planning

LLM-enhanced Scene Graph Learning for Household Rearrangement

AllenAct: A Framework for Embodied AI Research