Abstract:Active inference is a theory of perception, learning, and decision making that can be applied to neuroscience, robotics, psychology, and machine learning. Recently, intensive research has been taking place to scale up this framework using Monte Carlo tree search and deep learning. The goal of this activity is to solve more complicated tasks using deep active inference. First, we review the existing literature and then progressively build a deep active inference agent as follows: we (1) implement a variational autoencoder (VAE), (2) implement a deep hidden Markov model (HMM), and (3) implement a deep critical hidden Markov model (CHMM). For the CHMM, we implemented two versions, one minimizing expected free energy, CHMM[EFE] and one maximizing rewards, CHMM[reward]. Then we experimented with three different action selection strategies: the ε-greedy algorithm as well as softmax and best action selection. According to our experiments, the models able to solve the dSprites environment are the ones that maximize rewards. On further inspection, we found that the CHMM minimizing expected free energy almost always picks the same action, which makes it unable to solve the dSprites environment. In contrast, the CHMM maximizing reward keeps on selecting all the actions, enabling it to successfully solve the task. The only difference between those two CHMMs is the epistemic value, which aims to make the outputs of the transition and encoder networks as close as possible. Thus, the CHMM minimizing expected free energy repeatedly picks a single action and becomes an expert at predicting the future when selecting this action. This effectively makes the KL divergence between the output of the transition and encoder networks small. Additionally, when selecting the action down the average reward is zero, while for all the other actions, the expected reward will be negative. Therefore, if the CHMM has to stick to a single action to keep the KL divergence small, then the action down is the most rewarding. We also show in simulation that the epistemic value used in deep active inference can behave degenerately and in certain circumstances effectively lose, rather than gain, information. As the agent minimizing EFE is not able to explore its environment, the appropriate formulation of the epistemic value in deep active inference remains an open question.

End-to-End Pixel-Based Deep Active Inference for Body Perception and Action

Active inference body perception and action for humanoid robots

Neuroscience-inspired perception-action in robotics: applying active inference for state estimation, control and self-perception

Adaptive robot body learning and estimation through predictive coding

Active Vision for Robot Manipulators Using the Free Energy Principle

Active Inference for Learning and Development in Embodied Neuromorphic Agents

A Neural Active Inference Model of Perceptual-Motor Learning

Vision-Action Semantic Associative Learning Based on Spiking Neural Networks for Cognitive Robot.

Deconstructing deep active inference

Real-Time Biologically Inspired Action Recognition from Key Poses Using a Neuromorphic Architecture

The Free Energy Principle for Perception and Action: A Deep Learning Perspective

InfPose: Real-Time Infrared Multi-Human Pose Estimation for Edge Devices Based on Encoder-Decoder CNN Architecture

Sensorimotor learning for artificial body perception

Embodied Object Representation Learning and Recognition

The Active Inference Approach to Ecological Perception: General Information Dynamics for Natural and Artificial Embodied Cognition

Modeling human intention inference in continuous 3D domains by inverse planning and body kinematics

Active Sensing with Predictive Coding and Uncertainty Minimization

Object-Centric Scene Representations Using Active Inference

ALP: Action-Aware Embodied Learning for Perception

How Active Inference Could Help Revolutionise Robotics

Deconstructing Deep Active Inference: A Contrarian Information Gatherer