Abstract:Humans can perform complex tasks with long-term objectives by planning, reasoning, and forecasting outcomes of actions. For embodied agents to achieve similar capabilities, they must gain knowledge of the environment transferable to novel scenarios with a limited budget of additional trial and error. Learning-based approaches, such as deep RL, can discover and take advantage of inherent regularities and characteristics of the application domain from data, and continuously improve their performances, however at a cost of large amounts of training data. This thesis explores the development of data-driven techniques for spatial reasoning and planning tasks, focusing on enhancing learning efficiency, interpretability, and transferability across novel scenarios. Four key contributions are made. 1) CALVIN, a differential planner that learns interpretable models of the world for long-term planning. It successfully navigated partially observable 3D environments, such as mazes and indoor rooms, by learning the rewards and state transitions from expert demonstrations. 2) SOAP, an RL algorithm that discovers options unsupervised for long-horizon tasks. Options segment a task into subtasks and enable consistent execution of the subtask. SOAP showed robust performances on history-conditional corridor tasks as well as classical benchmarks such as Atari. 3) LangProp, a code optimisation framework using LLMs to solve embodied agent problems that require reasoning by treating code as learnable policies. The framework successfully generated interpretable code with comparable or superior performance to human-written experts in the CARLA autonomous driving benchmark. 4) Voggite, an embodied agent with a vision-to-action transformer backend that solves complex tasks in Minecraft. It achieved third place in the MineRL BASALT Competition by identifying action triggers to segment tasks into multiple stages.

The Landscape of Emerging AI Agent Architectures for Reasoning, Planning, and Tool Calling: A Survey

AI Planning: A Primer and Survey (Preliminary Report)

Software Architecture for Next-Generation AI Planning Systems

Revealing the Barriers of Language Agents in Planning

ReasonPlanner: Enhancing Autonomous Planning in Dynamic Environments with Temporal Knowledge Graphs and LLMs

An AI Planning-Based Approach to the Multi-Agent Plan Recognition Problem

Spatial Reasoning and Planning for Deep Embodied Agents

Towards an ASP-Based Architecture for Autonomous UAVs in Dynamic Environments (Extended Abstract)

Agent-E: From Autonomous Web Navigation to Foundational Design Principles in Agentic Systems

From SLAM to Situational Awareness: Challenges and Survey

ConceptAgent: LLM-Driven Precondition Grounding and Tree Search for Robust Task Planning and Execution

An ASP-Based Architecture for Autonomous UAVs in Dynamic Environments: Progress Report

Agent-Oriented Planning in Multi-Agent Systems

A Survey on Context-Aware Multi-Agent Systems: Techniques, Challenges and Future Directions

Agent architecture: a survey on RoboCup-99 simulator teams

Multi-Agent, Human-Agent and Beyond: A Survey on Cooperation in Social Dilemmas

A Domain-Independent Agent Architecture for Adaptive Operation in Evolving Open Worlds

Pangu-Agent: A Fine-Tunable Generalist Agent with Structured Reasoning

Design Patterns for Building More Efficient Generative Autonomous Agents: A Survey

LASP: Surveying the State-of-the-Art in Large Language Model-Assisted AI Planning