Predictive Perception of Sensorimotor Contingencies : Explaining perceptual presence and its absence in synaesthesia
A. Seth
Abstract:Does perception involve the deployment of predictive models conducting inference on the causes of sensory signals, along Bayesian-brain lines? Or does it depend on the skilful mastery of sensorimotor contingencies, as sensorimotor theories suggest? Here I describe a reconciliation of these distinct perspectives by the theory of Predictive Perception of Sensorimotor Contingencies (PPSMC). In PPSMC, generative models underlying perception incorporate explicitly counterfactual elements related to how sensory inputs would change on the basis of a broad repertoire of possible actions, even if these actions are not executed. These counterfactuallyextended generative models encode SMCs related to repertoires of sensorimotor dependencies. PPSMC extends predictive processing approaches to account for the phenomenology of ‘presence’ which, following sensorimotor theories, refers to the subjective reality of perceptual contents. PPSMC is also able, unlike sensorimotor theories, to account for the absence of perceptual presence in atypical cases like synaesthesia. Normal perception involves experiencing objects within perceptual scenes as real, as existing in the world. This property of “perceptual presence” has motivated “sensorimotor theories” which understand perception to involve the mastery of sensorimotor contingencies (SMCs) [1]. These ideas inherit from Gibsonian notions of “affordance” and from enactive cognitive science, both of which stress the importance of brainbody-world interactions in cognition, perception, and action [2]. On sensorimotor theory, the perception of (for example) a tomato as perceptually present is given by practical mastery of the SMCs governing how sensory responses elicited by the tomato will behave given specific actions (like eye movements). A strong point of this theory is that it suggests why there are differences in qualitative character between modalities, the reason being that different modalities instantiate different SMCs. However, sensorimotor theory faces two major challenges. The first is to specify at the level of neural mechanism what is meant by a SMC and by their mastery. The second is to account for instances of perception which apparently do not involve SMCs. Synaesthesia is a good example of the latter case. Graphemecolour synaesthetes, for example, have “concurrent” experiences of colour when viewing achromatic graphemic “inducer” stimuli [3]. Yet these inducer stimuli, by definition, do not engage SMCs associated with red objects. This poses a problem for sensorimotor theory. In addition, synaesthetic experiences typically lack perceptual presence: synaesthetes usually know 1 Sackler Centre for Consciousness Science, University of Sussex, UK. Email: a.k.seth@sussex.ac.uk that their concurrents are not actually part of the real world. Current theories of synaesthesia – like those suggesting “crossactivation” between brain regions involved in inducer and concurrent processing do not account for this critical phenomenological property. An alternative theoretical tradition sees the problem of perception as essentially one of inference about the causes of sensory signals. These “Bayesian brain” or “predictive processing” theories, which can be traced back to von Helmholtz in the 19th Century, are gaining increasing influence within cognitive neuroscience [4-6]. Here, the basic idea is that, in order to support adaptive responses, the brain must discover information about the likely external causes of sensory signals, without any direct access to these causes, using only information in the flux of the sensory signals themselves. Perception solves this problem via probabilistic, knowledge-driven inference on the causes of sensory signals. Applied to cortical networks, the concept of predictive processing overturns classical notions of perception as a largely “bottom-up” process of evidence accumulation or feature detection. Instead, predictive processing proposes that perceptual content is specified by top-down predictive signals emerging from multi-level hierarchicallyorganized generative models of the causes of sensory signals, which are continually modified by bottom-up prediction error signals communicating mismatches between predicted and actual signals across hierarchical levels. In this view, even low-level fine-grained perceptual content depends on a cascade of predictions flowing from very general abstract expectations which constrain successively more detailed predictions. While accumulating evidence is providing strong (though indirect) support for predictive processing theories, these theories have not so far addressed the key challenge of perceptual presence as identified within sensorimotor approaches. Neither have predictive processing accounts of synaesthesia yet been developed. Finally, sensorimotor and predictive processing theories have developed largely independently, with opportunities for their integration not fully appreciated. Here, I describe a new theoretical approach, Predictive Perception account of SensoriMotor Contingencies (PPSMC), which addresses these three challenges [7]. The core idea of PPSMC is that generative models underlying perception incorporate explicitly counterfactual elements related to how sensory inputs would change on the basis of a broad repertoire of possible actions, even if these actions are not executed. These counterfactually-extended generative models encode SMCs related to repertoires of sensorimotor dependencies. Critically, perceptual presence in PPSMC depends on the degree of counterfactual richness: A counterfactually-rich generative model will endow perceptual content with presence, while a counterfactually-poor model will result in perceptual content lacking in presence. PPSMC offers a number of innovations as compared to sensorimotor or predictive processing approaches considered separately. First, the concept of a counterfactually-rich generative model provides a neurofunctional operationalization of the “mastery of sensorimotor contingencies” central to sensorimotor theory. Second, it extends predictive processing to account for the fundamental phenomenological dimension of perceptual presence. Third, it suggests a solution to the challenge presented by synaesthesia: While the generative models underlying normal perception are typically counterfactually rich (reflecting a large repertoire of possible sensorimotor dependencies), those underlying synaesthetic concurrents are hypothesized to be counterfactually poor. Fourth, the theory naturally accommodates phenomenological differences between a range of experiential states including dreaming, hallucination, and the like. And finally it may enable a new view of the phenomenological (in)determinacy of normal perception.