Abstract:Humans and non-human animals show great flexibility in spatial navigation, including the ability to return to specific locations based on as few as one single experience. To study spatial navigation in the laboratory, watermaze tasks, in which rats have to find a hidden platform in a pool of cloudy water surrounded by spatial cues, have long been used. Analogous tasks have been developed for human participants using virtual environments. Spatial learning in the watermaze is facilitated by the hippocampus. In particular, rapid, one-trial, allocentric place learning, as measured in the delayed-matching-to-place variant of the watermaze task, which requires rodents to learn repeatedly new locations in a familiar environment, is hippocampal dependent. In this article, we review some computational principles, embedded within a reinforcement learning framework, that utilise hippocampal spatial representations for navigation in watermaze tasks. We consider which key elements underlie their efficacy, and discuss their limitations in accounting for hippocampus-dependent navigation, both in terms of behavioural performance (i.e. how well do they reproduce behavioural measures of rapid place learning) and neurobiological realism (i.e. how well do they map to neurobiological substrates involved in rapid place learning). We discuss how an actor-critic architecture, enabling simultaneous assessment of the value of the current location and of the optimal direction to follow, can reproduce one-trial place learning performance as shown on watermaze and virtual delayed-matching-to-place tasks by rats and humans, respectively, if complemented with map-like place representations. The contribution of actor-critic mechanisms to delayed-matching-to-place performance is consistent with neurobiological findings implicating the striatum and hippocampo-striatal interaction in delayed-matching-to-place performance, given that the striatum has been associated with actor-critic mechanisms. Moreover, we illustrate that hierarchical computations embedded within an actor-critic architecture may help to account for aspects of flexible spatial navigation. The hierarchical reinforcement learning approach separates trajectory control via a temporal-difference error from goal selection via a goal prediction error and may account for flexible, trial-specific, navigation to familiar goal locations, as required in some arm-maze place memory tasks, although it does not capture one-trial learning of new goal locations, as observed in open field, including watermaze and virtual, delayed-matching-to-place tasks. Future models of one-shot learning of new goal locations, as observed on delayed-matching-to-place tasks, should incorporate hippocampal plasticity mechanisms that integrate new goal information with allocentric place representation, as such mechanisms are supported by substantial empirical evidence.

Developments of a water-maze procedure for studying spatial learning in the rat

Lessons in cognition: A review of maze designs and procedures used to measure spatial learning in fish

Auditory Cue Induced Place Navigation in Rats

En route to delineating hippocampal roles in spatial learning

A novel behavioral apparatus for spontaneous exploration and operant conditioning of social information under spatial conditions in rats

Reinforcement learning approaches to hippocampus-dependent flexible spatial navigation

A generalised framework for detailed classification of swimming paths inside the Morris Water Maze

Rodent maze studies: from following simple rules to complex map learning

Looking beyond the standard version of the Morris water task in the assessment of mouse models of cognitive deficits

The Prominence of Action Sequences and Behavioral Similarity in the Morris Water Task

Investigating Navigation Strategies in the Morris Water Maze through Deep Reinforcement Learning

A Novel Automated Touchscreen Procedure for Assessing Learning in the Rat Using Computer Graphic Stimuli

The Active Place Avoidance (APA) Test, an Effective, Versatile and Repeatable Spatial Learning Task for Mice

Hippocampal development and the dissociation of cognitive-spatial mapping from motor performance

Hippocampal-dependent navigation in head-fixed mice using a floating real-world environment

The Effects of AMPA‐induced Lesions of the Septo‐hippocampal Cholinergic Projection on Aversive Conditioning to Explicit and Contextual Cues and Spatial Learning in the Water Maze

Renovating the Barnes maze for mouse models of Dementia with STARR FIELD: A 4-day protocol that probes learning rate, retention and cognitive flexibility.

Protocol to investigate the gradual selection and deployment of goal-oriented search strategies during unsupervised navigation in mice

Adult zebrafish can learn Morris water maze-like tasks in a two-dimensional virtual reality system

Spatial Learning by Using Non-Visual Geometry and a Visual 3D Landmark in Zebrafish (Danio rerio)

A transformer-based deep reinforcement learning approach to spatial navigation in a partially observable Morris Water Maze