Abstract:The goal of the present paper is to provide an embodied cognitive science view on representation. Using the fundamental task of category learning, we will demonstrate that this perspective enables us to shed new light on many pertinent issues and opens up new prospects for investigation. The main focus of this paper is on the prerequisites to acquire representations of objects in the real world. We suggest that the main prerequisite is embodiment which allows an agent--human, animal or robot--to manipulate its sensory input such that invariances are generated. These invariances, in turn, are the basis of representation formation. In other words, the paper does not focus on representations per se, but rather discusses the various processes involved in order to make learning and representation acquisition possible. The argument structure is as follows. First we introduce two new perspectives on representation, namely frame-of-reference, and complete agent. Then we elaborate the complete agent perspective and focus in particular on embodiment and situatedness. We argue that embodiment has two main aspects, a dynamic and an information theoretic one. Focusing on the latter, there are a number of implications: Representation can only be understood if the embedding of the neural substrate in the physical agent is known, which includes morphology (shape), positioning and nature of sensors. Because an autonomous mobile agent in the real world is exposed to a continuously changing high-dimensional stream of sensory stimulation, if it is to learn category distinctions, it first needs a focus of attention mechanism, and then it must have a way to reduce the dimensionality of this high-dimensional sensory stream. Learning is very hard because the invariances are typically not found in the sensory data directly--the classical problem of object constancy: it is a so-called type 2 problem. Rather than trying to improve the learning algorithms--which is the standard approach--the embodied cognitive science view suggests a different approach which focuses on the nature of the data: the agent is not passively exposed to a given data distribution, but, by exploiting its body and through the interaction with the environment, it can actually generate the data. More specifically, it can generate correlated data that has the property that it can be easily learned. This learnability is due to redundancies resulting from the appropriate interactions with the environment. Through such interactions, the former type 2 problem is transformed into a type 1 problem, thus reducing the complexity of the learning task by orders of magnitude. By observing the frame-of-reference problem we will discuss to what extent these invariances are reflected--represented--in the "neural substrate", i.e. the internal mechanisms of the agent. It is concluded, that representation is not a concept that can be studied in the abstract, but should be elaborated in the context of concrete agent-environment interactions. These ideas are all illustrated with examples of natural agents and artificial agents. In particular, we will present a suite of experiments on simulated and real-world artificial agents instantiating the main arguments.

Dual Representation and the Linking of Concrete and Symbolic Representations

The cost of multiple representations: Learning number symbols with abstract and concrete representations.

Arguments about the nature of concepts: Symbols, embodiment, and beyond

Symbolic representation in early years learning: The acquisition of complex notions

Symbolic functioning in preschool children

Dual Embodied-Symbolic Concept Representations for Deep Learning

Representation and Development of Cognition

Mental Representations: A Dual Coding Approach

DeFT: A conceptual framework for considering learning with multiple representations

Representation in natural and artificial agents: an embodied cognitive science perspective

Symbolic Representation across Domains in Preschool Children

Mutual Bootstrapping Between Language and Analogical Processing

Impact of concreteness on comprehensibility, interest, and memory for text: Implications for dual coding theory and text design.

Representing the World in Language and Thought

Language systematizes attention: How relational language enhances relational representation by guiding attention

The material features of multiple representations and their cognitive and social affordances for science understanding

Spatiotemporal dynamics of abstract and concrete semantic representations

Symbol Grounding or the Emergence of Symbols? Vocabulary Growth in Children and a Connectionist Net

Embodied Cognition in Education: Possibilities and Limitations of Hybrid Representations

Representationalism and Power: The Individual Subject and Distributed Cognition in the Field of Educational Technology

Where Hypotheses Come From: Learning New Relations by Structural Alignment