Abstract:A key challenge for AI is to build embodied systems that operate in dynamically changing environments. Such systems must adapt to changing task contexts and learn continuously. Although standard deep learning systems achieve state of the art results on static benchmarks, they often struggle in dynamic scenarios. In these settings, error signals from multiple contexts can interfere with one another, ultimately leading to a phenomenon known as catastrophic forgetting. In this article we investigate biologically inspired architectures as solutions to these problems. Specifically, we show that the biophysical properties of dendrites and local inhibitory systems enable networks to dynamically restrict and route information in a context-specific manner. Our key contributions are as follows: first, we propose a novel artificial neural network architecture that incorporates active dendrites and sparse representations into the standard deep learning framework. Next, we study the performance of this architecture on two separate benchmarks requiring task-based adaptation: Meta-World, a multi-task reinforcement learning environment where a robotic agent must learn to solve a variety of manipulation tasks simultaneously; and a continual learning benchmark in which the model's prediction task changes throughout training. Analysis on both benchmarks demonstrates the emergence of overlapping but distinct and sparse subnetworks, allowing the system to fluidly learn multiple tasks with minimal forgetting. Our neural implementation marks the first time a single architecture has achieved competitive results in both multi-task and continual learning settings. Our research sheds light on how biological properties of neurons can inform deep learning systems to address dynamic scenarios that are typically impossible for traditional ANNs to solve.

Combined Model for Partially-Observable and Non-Observable Task Switching: Solving Hierarchical Reinforcement Learning Problems Statically and Dynamically with Transfer Learning

Adaptive coordination of working-memory and reinforcement learning in non-human primates performing a trial-and-error problem solving task

Evolving hierarchical memory-prediction machines in multi-task reinforcement learning

Switching Attention in Time-Varying Environments via Bayesian Inference of Abstractions

Partially Observable Planning and Learning for Systems with Non-Uniform Dynamics

Future shapes present: autonomous goal-directed and sensory-focused mode switching in a Bayesian allostatic network model

Adaptive Robot Assistance: Expertise and Influence in Multi-User Task Planning

Hierarchical Orchestra of Policies

Hybrid Recurrent Models Support Emergent Descriptions for Hierarchical Planning and Control

Multi-Horizon Representations with Hierarchical Forward Models for Reinforcement Learning

Modular transfer learning with transition mismatch compensation for excessive disturbance rejection

Lifelong Reinforcement Learning via Neuromodulation

Recurrent mutation in the PIEZO1 gene in two families of hereditary xerocytosis with fetal hydrops

Optimal Decision-Making in Mixed-Agent Partially Observable Stochastic Environments via Reinforcement Learning

An unsupervised autonomous learning framework for goal-directed behaviours in dynamic contexts

Hierarchical LLMs In-the-loop Optimization for Real-time Multi-Robot Target Tracking under Unknown Hazards

TOM: Learning Policy-Aware Models for Model-Based Reinforcement Learning via Transition Occupancy Matching

Real-Time Recurrent Reinforcement Learning

Leveraging Knowledge Graph-Based Human-Like Memory Systems to Solve Partially Observable Markov Decision Processes

Avoiding Catastrophe: Active Dendrites Enable Multi-Task Learning in Dynamic Environments

Inhomogeneous metallic phase in a disordered Mott insulator in two dimensions.