Abstract:Machines that can replicate human intelligence with type 2 reasoning capabilities should be able to reason at multiple levels of spatio-temporal abstractions and scales using internal world models. Devising formalisms to develop such internal world models, which accurately reflect the causal hierarchies inherent in the dynamics of the real world, is a critical research challenge in the domains of artificial intelligence and machine learning. This thesis identifies several limitations with the prevalent use of state space models (SSMs) as internal world models and propose two new probabilistic formalisms namely Hidden-Parameter SSMs and Multi-Time Scale SSMs to address these drawbacks. The structure of graphical models in both formalisms facilitates scalable exact probabilistic inference using belief propagation, as well as end-to-end learning via backpropagation through time. This approach permits the development of scalable, adaptive hierarchical world models capable of representing nonstationary dynamics across multiple temporal abstractions and scales. Moreover, these probabilistic formalisms integrate the concept of uncertainty in world states, thus improving the system's capacity to emulate the stochastic nature of the real world and quantify the confidence in its predictions. The thesis also discuss how these formalisms are in line with related neuroscience literature on Bayesian brain hypothesis and predicitive processing. Our experiments on various real and simulated robots demonstrate that our formalisms can match and in many cases exceed the performance of contemporary transformer variants in making long-range future predictions. We conclude the thesis by reflecting on the limitations of our current models and suggesting directions for future research.

Learning symbolic models of stochastic domains

Learning Probabilistic Relational Planning Rules.

Learning planning rules in noisy stochastic worlds

Learning STRIPS operators from noisy and incomplete observations

Learning to Model the World with Language

Learning probabilistic relational dynamics for multiple tasks

Stochastic Games for Interactive Manipulation Domains

Model Learning and Knowledge Sharing for Cooperative Multiagent Systems in Stochastic Environment

Learning Abstract World Model for Value-preserving Planning with Options

Epistemic Exploration for Generalizable Planning and Learning in Non-Stationary Settings

From Reals to Logic and Back: Inventing Symbolic Vocabularies, Actions and Models for Planning from Raw Data

Learning World Models With Hierarchical Temporal Abstractions: A Probabilistic Perspective

Probabilistic World Modeling with Asymmetric Distance Measure

Learning Type-Generalized Actions for Symbolic Planning

Learning Symbolic Operators for Task and Motion Planning

Learning Neuro-Symbolic Relational Transition Models for Bilevel Planning

Learning model-based planning from scratch

Non-cooperative Multi-agent Systems with Exploring Agents

Learning to Operate in Open Worlds by Adapting Planning Models

Nested Reasoning About Autonomous Agents Using Probabilistic Programs

Learning Neuro-Symbolic Skills for Bilevel Planning