Abstract:The patch-leaving problem is a canonical foraging task, in which a forager must decide to leave a current resource in search for another. Theoretical work has derived optimal strategies for when to leave a patch, and experiments have tested for conditions where animals do or do not follow an optimal strategy. Nevertheless, models of patch-leaving decisions do not consider the imperfect and noisy sampling process through which an animal gathers information, and how this process is constrained by neurobiological mechanisms. In this theoretical study, we formulate an evidence accumulation model of patch-leaving decisions where the animal averages over noisy measurements to estimate the state of the current patch and the overall environment. We solve the model for conditions where foraging decisions are optimal and equivalent to the marginal value theorem, and perform simulations to analyze deviations from optimal when these conditions are not met. By adjusting the drift rate and decision threshold, the model can represent different "strategies", for example an incremental, decremental, or counting strategy. These strategies yield identical decisions in the limiting case but differ in how patch residence times adapt when the foraging environment is uncertain. To describe sub-optimal decisions, we introduce an energy-dependent marginal utility function that predicts longer than optimal patch residence times when food is plentiful. Our model provides a quantitative connection between ecological models of foraging behavior and evidence accumulation models of decision making. Moreover, it provides a theoretical framework for potential experiments which seek to identify neural circuits underlying patch-leaving decisions.Foraging is a ubiquitous animal behavior, performed by organisms as different as worms, birds, rats, and humans. Although the behavior has been extensively studied, it is not known how the brain processes information obtained during foraging activity to make subsequent foraging decisions. We form an evidence accumulation model of foraging decisions that describes the process through which an animal gathers information and uses it to make foraging decisions. By building on studies of the neural decision mechanisms within systems neuroscience, this model connects the foraging decision process with ecological models of patch-leaving decisions, such as the marginal value theorem. The model suggests the existence of different foraging strategies, which optimize for different environmental conditions and their potential implementation by neural decision making circuits. The model also shows how state-dependence, such as satiation level, can affect evidence accumulation to lead to sub-optimal foraging decisions. Our model provides a framework for future experimental studies which seek to elucidate how neural decision making mechanisms have been shaped by evolutionary forces in an animal's surrounding environment.

Random Search with Memory in Patchy Media: Exploration-Exploitation Tradeoff

A random walk model to study the cycles emerging from the exploration-exploitation trade-off

Exploration-exploitation trade-off features a saltatory search behaviour

Foraging Under Uncertainty Follows the Marginal Value Theorem with Bayesian Updating of Environment Representations

Normative theory of patch foraging decisions

Efficient search by optimized intermittent random walks

Differential patch-leaving behavior during probabilistic foraging in humans and gerbils

Exploration by Running Away from the Past

Social Learning and the Exploration-Exploitation Tradeoff

Exploration Unbound

Randomized Exploration in Cooperative Multi-Agent Reinforcement Learning

Learning and choosing in an uncertain world: An investigation of the explore-exploit dilemma in static and dynamic environments

The Exploration-Exploitation Dilemma Revisited: An Entropy Perspective

Stochastic dynamics of social patch foraging decisions

Gamifying optimization: a Wasserstein distance-based analysis of human search

Foraging as an evidence accumulation process

Curiosity in Hindsight: Intrinsic Exploration in Stochastic Environments

Mean Field Equilibria for Competitive Exploration in Resource Sharing Settings

Approximate information for efficient exploration-exploitation strategies

Subspace-Aware Exploration for Sparse-Reward Multi-Agent Tasks.

Disentangling Exploration from Exploitation