Abstract:Humans can perform complex tasks with long-term objectives by planning, reasoning, and forecasting outcomes of actions. For embodied agents to achieve similar capabilities, they must gain knowledge of the environment transferable to novel scenarios with a limited budget of additional trial and error. Learning-based approaches, such as deep RL, can discover and take advantage of inherent regularities and characteristics of the application domain from data, and continuously improve their performances, however at a cost of large amounts of training data. This thesis explores the development of data-driven techniques for spatial reasoning and planning tasks, focusing on enhancing learning efficiency, interpretability, and transferability across novel scenarios. Four key contributions are made. 1) CALVIN, a differential planner that learns interpretable models of the world for long-term planning. It successfully navigated partially observable 3D environments, such as mazes and indoor rooms, by learning the rewards and state transitions from expert demonstrations. 2) SOAP, an RL algorithm that discovers options unsupervised for long-horizon tasks. Options segment a task into subtasks and enable consistent execution of the subtask. SOAP showed robust performances on history-conditional corridor tasks as well as classical benchmarks such as Atari. 3) LangProp, a code optimisation framework using LLMs to solve embodied agent problems that require reasoning by treating code as learnable policies. The framework successfully generated interpretable code with comparable or superior performance to human-written experts in the CARLA autonomous driving benchmark. 4) Voggite, an embodied agent with a vision-to-action transformer backend that solves complex tasks in Minecraft. It achieved third place in the MineRL BASALT Competition by identifying action triggers to segment tasks into multiple stages.

Reasoning With Scene Graphs for Robot Planning Under Partial Observability

iCORPP: Interleaved commonsense reasoning and probabilistic planning on robots

EmbodiedRAG: Dynamic 3D Scene Graph Retrieval for Efficient and Scalable Robot Task Planning

A User Interface for Sense-making of the Reasoning Process while Interacting with Robots

Robot Active Neural Sensing and Planning in Unknown Cluttered Environments

Spatial Reasoning and Planning for Deep Embodied Agents

VeriGraph: Scene Graphs for Execution Verifiable Robot Planning

Online Grounding of Symbolic Planning Domains in Unknown Environments

Online Replanning in Belief Space for Partially Observable Task and Motion Problems

Elephants Don't Pack Groceries: Robot Task Planning for Low Entropy Belief States

Long-term Robot Manipulation Task Planning with Scene Graph and Semantic Knowledge

Situational Awareness Matters in 3D Vision Language Reasoning

Planning under periodic observations: bounds and bounding-based solutions

Scene Reconstruction with Functional Objects for Robot Autonomy

Anytime Game-Theoretic Planning with Active Reasoning about Humans' Latent States for Human-Centered Robots

SG-Bot: Object Rearrangement via Coarse-to-Fine Robotic Imagination on Scene Graphs

Task and Motion Planning in Hierarchical 3D Scene Graphs

SayPlan: Grounding Large Language Models using 3D Scene Graphs for Scalable Robot Task Planning

Reasoning about the Unseen for Efficient Outdoor Object Navigation

Learning to reason over scene graphs: a case study of finetuning GPT-2 into a robot language model for grounded task planning

Temporal Logic Robot Task Planning with Active Acquisition of Information.