A model of memory, learning and recognition

Bruce Hoeneisen

DOI: https://doi.org/10.48550/arXiv.physics/0210093

2002-10-24

Abstract:We propose a simple model of recognition, short-term memory, long-term memory and learning.

Biological Physics,Quantitative Biology

What problem does this paper attempt to address?

How the Brain Formulates Memory: A Spatio-Temporal Model Research Frontier.

Jun Hu,Huajin Tang,K. C. Tan,Haizhou Li

DOI: https://doi.org/10.1109/mci.2016.2532268

IF: 9.809

2016-01-01

IEEE Computational Intelligence Magazine

Abstract:Memory is a complex process across different brain regions and a fundamental function for many cognitive behaviors. Emerging experimental results suggest that memories are represented by populations of neurons and organized in a categorical and hierarchical manner. However, it is still not clear how the neural mechanisms are emulated in computational models. In this paper, we present a spatio-temporal memory (STM) model using spiking neurons to explore the memory formulation and organization in the brain. Unlike previous approaches, this model employs temporal population codes as the neural representation of information and spike-timing-based learning methods to formulate the memory structure. It explicitly demonstrates that the complex spatio-temporal patterns are the internal neural representations of memory items. Two types of memory processes are analyzed and emulated: associative memory, i.e., spatio-temporal patterns driven by intra-assembly connections, and episodic memory, i.e., temporally separated spatio-temporal patterns linked by inter-assembly connections. Our model will provide a computational substrate based on low-level neural circuits for developing neuromorphic cognitive systems with wide applications.
Brain Inspired Cognitive System for Learning and Memory

Huajin Tang,Weiwei Huang

DOI: https://doi.org/10.1007/978-3-642-24955-6_57

2011-01-01

Abstract:Hippocampus, a major component of brain, plays an important role in learning and memory. In this paper, we present our brain-inspired cognitive model which combines a hippocampal circuitry together with hierarchical vision architecture. The structure of the simulated hippocampus is designed based on an approximate mammalian neuroanatomy. The connectivity between neural areas is based on known anatomical measurements. The proposed model could be used to explore the memory property and its corresponding neuron activities in hippocampus. In our simulation test, the model shows the ability of recalling character images that it had been learned before.
A Formal Approach to Modeling the Memory of a Living Organism

Dan Guralnik

DOI: https://doi.org/10.48550/arXiv.1003.3821

2010-03-19

Abstract:We consider a living organism as an observer of the evolution of its environment recording sensory information about the state space X of the environment in real time. Sensory information is sampled and then processed on two levels. On the biological level, the organism serves as an evaluation mechanism of the subjective relevance of the incoming data to the observer: the observer assigns excitation values to events in X it could recognize using its sensory equipment. On the algorithmic level, sensory input is used for updating a database, the memory of the observer whose purpose is to serve as a geometric/combinatorial model of X, whose nodes are weighted by the excitation values produced by the evaluation mechanism. These values serve as a guidance system for deciding how the database should transform as observation data mounts. We define a searching problem for the proposed model and discuss the model's flexibility and its computational efficiency, as well as the possibility of implementing it as a dynamic network of neuron-like units. We show how various easily observable properties of the human memory and thought process can be explained within the framework of this model. These include: reasoning (with efficiency bounds), errors, temporary and permanent loss of information. We are also able to define general learning problems in terms of the new model, such as the language acquisition problem.

Artificial Intelligence,Data Structures and Algorithms,Machine Learning,Neurons and Cognition
A Neural Dynamic Model based on Activation Diffusion and a Micro-Explanation for Cognitive Operations

Hui Wei

DOI: https://doi.org/10.48550/arXiv.2012.00104

2020-11-27

Abstract:The neural mechanism of memory has a very close relation with the problem of representation in artificial intelligence. In this paper a computational model was proposed to simulate the network of neurons in brain and how they process information. The model refers to morphological and electrophysiological characteristics of neural information processing, and is based on the assumption that neurons encode their firing sequence. The network structure, functions for neural encoding at different stages, the representation of stimuli in memory, and an algorithm to form a memory were presented. It also analyzed the stability and recall rate for learning and the capacity of memory. Because neural dynamic processes, one succeeding another, achieve a neuron-level and coherent form by which information is represented and processed, it may facilitate examination of various branches of Artificial Intelligence, such as inference, problem solving, pattern recognition, natural language processing and learning. The processes of cognitive manipulation occurring in intelligent behavior have a consistent representation while all being modeled from the perspective of computational neuroscience. Thus, the dynamics of neurons make it possible to explain the inner mechanisms of different intelligent behaviors by a unified model of cognitive architecture at a micro-level.

Neurons and Cognition,Artificial Intelligence
A Model of the Hippocampal–cortical Memory System

Xiaochuan Pan,Minoru Tsukada

DOI: https://doi.org/10.1007/s00422-006-0074-8

IF: 3.072

2006-01-01

Biological Cybernetics

Abstract:Based on physiological evidence, we propose a theoretical model of the hippocampal–cortical memory system. The model consists of the following components: the sensory system, the hippocampus (short-term memory), and the association cortex (long-term memory). A series of key codes (local information) is supplied from the sensory system, while context (global information) is inputted from the hippocampus. The two inputs interact dynamically in the association cortex. The interactive neurons work as a detector of coincidence. The cortical network learns the memory information through the coincidence window and, finally, stores it in the form of attractors. This local–global information works as an addressor to designate the stored location of the memory in the association cortex and accelerates the process of storing and retrieving memory information.
A Hierarchically Organized Memory Model With Temporal Population Coding

Qiang Yu,Huajin Tang,Jun Hu,Kay Chen Tan

DOI: https://doi.org/10.1007/978-3-319-55310-8_7

2017-01-01

Abstract:Memory is a critical process in the brain to many cognitive behaviors. It is a complex process operating across different brain regions. However, the organizing principles of memory systems remain unclear. Emerging experiment results show that memories are represented by population of neurons and organized in a categorical and hierarchical manner. In this work, we describe a hierarchically organized memory (HOM) model using spiking neurons, in which temporal population codes are considered as the neural representation of information and spike-timing-based learning methods are employed to train the network. The results have demonstrated that memory coding units are formed into neural cliques, and information are stored in the form of associative memory within gamma cycles. Moreover, temporally separated patterns can be linked and compressed via enhanced connections among neural groups forming episodic memory. Our model provides a computational interpretation of memory organization at a system level.
Discrete-state and continuous models of recognition memory: Testing core properties under minimal assumptions.

David Kellen,Karl Christoph Klauer

DOI: https://doi.org/10.1037/xlm0000016

2014-01-01

Abstract:A classic discussion in the recognition-memory literature concerns the question of whether recognition judgments are better described by continuous or discrete processes. These two hypotheses are instantiated by the signal detection theory model (SDT) and the 2-high-threshold model, respectively. Their comparison has almost invariably relied on receiver operating characteristic data. A new model-comparison approach based on ranking judgments is proposed here. This approach has several advantages: It does not rely on particular distributional assumptions for the models, and it does not require costly experimental manipulations. These features permit the comparison of the models by means of simple paired-comparison tests instead of goodness-of-fit results and complex model-selection methods that are predicated on many auxiliary assumptions. Empirical results from 2 experiments are consistent with a continuous memory process such as the one assumed by SDT.

psychology, experimental
Introducing Memory and Association Mechanism into a Biologically Inspired Visual Model

Qiao Hong,Li Yinlin,Tang Tang,Wang Peng

DOI: https://doi.org/10.48550/arXiv.1307.1388

2013-07-05

Abstract:A famous biologically inspired hierarchical model firstly proposed by Riesenhuber and Poggio has been successfully applied to multiple visual recognition tasks. The model is able to achieve a set of position- and scale-tolerant recognition, which is a central problem in pattern recognition. In this paper, based on some other biological experimental results, we introduce the Memory and Association Mechanisms into the above biologically inspired model. The main motivations of the work are (a) to mimic the active memory and association mechanism and add the 'top down' adjustment to the above biologically inspired hierarchical model and (b) to build up an algorithm which can save the space and keep a good recognition performance. The new model is also applied to object recognition processes. The primary experimental results show that our method is efficient with much less memory requirement.

Artificial Intelligence
Memory networks

Jason Weston, Sumit Chopra, Antoine Bordes

2014-10-15

Abstract:We describe a new class of learning models called memory networks. Memory networks reason with inference components combined with a long-term memory component; they learn how to use these jointly. The long-term memory can be read and written to, with the goal of using it for prediction. We investigate these models in the context of question answering (QA) where the long-term memory effectively acts as a (dynamic) knowledge base, and the output is a textual response. We evaluate them on a large-scale QA task, and a smaller, but more complex, toy task generated from a simulated world. In the latter, we show the reasoning power of such models by chaining multiple supporting sentences to answer questions that require understanding the intension of verbs.
A Cognitive Model Based Framework and Multi-layer Storage Architecture for Associative Memory

Jiandong Li,Runhe Huang,Kevin I-Kai Wang

DOI: https://doi.org/10.1109/ICCICC50026.2020.9450213

2020-01-01

Abstract:Memory is the foundation of intelligence. KID model covers multiple human cognitive processes such as learning and memory. This paper refines its memory process, especially focusing on associative memory of long-term memory. An associative memory framework with novel neural network storage architectures is presented to simulate human-like associative memory ability for machine intelligence. The pr...
A New Pattern Associative Memory Model for Image Recognition Based on Hebb Rules and Dot Product

Mingyue Gao,Limiao Deng,Yanjiang Wang

DOI: https://doi.org/10.1117/12.2302483

2018-01-01

Abstract:A great number of associative memory models have been proposed to realize information storage and retrieval inspired by human brain in the last few years. However, there is still much room for improvement for those models. In this paper, we extend a binary pattern associative memory model to accomplish real-world image recognition. The learning process is based on the fundamental Hebb rules and the retrieval is implemented by a normalized dot product operation. Our proposed model can not only fulfill rapid memory storage and retrieval for visual information but also have the ability on incremental learning without destroying the previous learned information. Experimental results demonstrate that our model outperforms the existing Self-Organizing Incremental Neural Network (SOINN) and Back Propagation Neuron Network (BPNN) on recognition accuracy and time efficiency.
A critical comparison of discrete-state and continuous models of recognition memory: Implications for recognition and beyond.

Angela M. Pazzaglia,Chad Dube,Caren M. Rotello

DOI: https://doi.org/10.1037/a0033044

IF: 22.4

2013-01-01

Psychological Bulletin

Abstract:Multinomial processing tree (MPT) models such as the single high-threshold, double high-threshold, and low-threshold models are discrete-state decision models that map internal cognitive events onto overt responses. The apparent benefit of these models is that they provide independent measures of accuracy and response bias, a claim that has motivated their frequent application in many areas of psychological science including perception, item and source memory, social cognition, reasoning, educational testing, eyewitness testimony, and psychopathology. Before appropriate conclusions about a given analysis can be drawn, however, one must first confirm that the model's assumptions about the underlying structure of the data are valid. The current review outlines the assumptions of several popular MPT models and assesses their validity using multiple sources of evidence, including receiver operating characteristics, direct model fits, and experimental tests of qualitative predictions. We argue that the majority of the evidence is inconsistent with these models and that, instead, the evidence supports continuous models such as those based on signal detection theory (SDT). Hybrid models that incorporate both SDT and MPT processes are also explored, and we conclude that these models retain the limitations associated with their threshold model predecessors. The potentially severe consequences associated with using an invalid model to interpret data are discussed, and a simple tutorial and model-fitting tool is provided to allow implementation of the empirically supported SDT model.

psychology, multidisciplinary
Simple Models of Plant Learning and Memory

Indrani Bose,Rajesh Karmakar

DOI: https://doi.org/10.1238/Physica.Topical.106a00009

2003-07-01

Abstract:Plants are capable of intelligent responses to complex environmental signals. Learning and memory play fundamental roles in such responses. Two simple models of plant memory are proposed based on the calcium-signalling system. The memory states correspond to steady state distributions of calcium ions.

Condensed Matter,Quantitative Biology
Hippocampal CA3–CA1 Synaptic Network Model of Memory

Lei Yang,Hong-Hui Zhang,Zhong-Kui Sun,Lin Du,Guan-Rong Chen

DOI: https://doi.org/10.1007/s11071-024-09375-4

IF: 5.741

2024-01-01

Nonlinear Dynamics

Abstract:A synaptic network model is proposed to characterize both short-term and long-term memory. The model describes the calcium concentration-dependent bistability of CaMKII phosphorylation levels and the transition of CaMKII between DOWN and UP states under different stimulation protocols. Numerical results demonstrate that short-term memory can be characterized by a trajectory tending to zero, while long-term memory can be characterized by a heteroclinic trajectory based on the principle of winnerless competition. Additionally, the results suggest that the model forms memories more efficiently with faster and longer training.
A Model of the Interaction Between Mood and Memory

E. T. Rolls,S. M. Stringer

DOI: https://doi.org/10.1080/713663216

2001-01-01

Network Computation in Neural Systems

Abstract:This paper investigates a neural network model of the interaction between mood and memory. The model has two attractor networks that represent the inferior temporal cortex (IT), which stores representations of visual stimuli, and the amygdala, the activity of which reflects the mood state. The two attractor networks are coupled by forward and backward projections. The model is however generic, and is relevant to understanding the interaction between different pairs of modules in the brain, particularly, as is the case with moods and memories, when there are fewer states represented in one module than in the other. During learning, a large number of patterns are presented to the IT, each paired with one of two mood states represented in the amygdala. The recurrent connections within each module, the forward connections from the memory module to the amygdala, and the backward connections from the amygdala to the memory module, are associatively modified. It is shown how the mood state in the amygdala can influence which memory patterns are recalled in the memory module. Further, it is shown that if there is an existing mood state in the amygdala, it can be difficult to change it even when a retrieval cue is presented to the memory module that is associated with a different mood state. It is also shown that the backprojections from the amygdala to the memory module must be relatively weak if memory retrieval in the memory module is not to be disrupted. The results are relevant to understanding the interaction between structures important in mood and emotion (such as the amygdala and orbitofrontal cortex) and other brain areas involved in storing objects and faces (such as the inferior temporal visual cortex) and memories (such as the hippocampus).
A Machine with Short-Term, Episodic, and Semantic Memory Systems

Taewoon Kim,Michael Cochez,Vincent François-Lavet,Mark Neerincx,Piek Vossen

DOI: https://doi.org/10.1609/aaai.v37i1.25075

2024-08-19

Abstract:Inspired by the cognitive science theory of the explicit human memory systems, we have modeled an agent with short-term, episodic, and semantic memory systems, each of which is modeled with a knowledge graph. To evaluate this system and analyze the behavior of this agent, we designed and released our own reinforcement learning agent environment, "the Room", where an agent has to learn how to encode, store, and retrieve memories to maximize its return by answering questions. We show that our deep Q-learning based agent successfully learns whether a short-term memory should be forgotten, or rather be stored in the episodic or semantic memory systems. Our experiments indicate that an agent with human-like memory systems can outperform an agent without this memory structure in the environment.

Artificial Intelligence
A Full-Neuron Memory Model Designed for Neuromorphic Systems

Kefei Liu,Xiaoxin Cui,Chenglong Zou,Yisong Kuang,Yi Zhong,Kanglin Xiao,Yuan Wang

DOI: https://doi.org/10.1109/aicas54282.2022.9869902

2022-01-01

Abstract:Neuromorphic computing is considered to be a promising approach to developing general artificial intelligence. Like a human brain, the neuromorphic system uses neurons as the basic unit for information computing and storage. Neural networks have made great progress in some intelligent tasks by borrowing from the way the brain computes. Another important part of forming artificial intelligence is memory storage in neuromorphic systems. However, there is a lack of effective and practical methods for information storing in neuromorphic systems. In this paper, we propose a brain-like full-neuron memory (FNM) model to store memories in neuromorphic systems. We successfully stored ten numbers and letters in the FNM model. Memory objects are learned through Hebbian synaptic plasticity in an unsupervised manner and stored as neural activations in FNM. FNM works in a content-addressed way, which means the stored numbers and letters can be recalled by associative either visual or auditory stimuli. Memories are hierarchically structured in FNM. The numbers and letters can be recalled in full by incomplete input stimuli benefiting from the hierarchical structure. Some experiments are designed to verify the memory ability of the FNM model. With FNM model, the complete information processing and storage can be done in neuromorphic systems without taking the intermediate results out and processing or storing them on other computing platforms.
Schrodinger's Memory: Large Language Models

Wei Wang,Qing Li

2024-09-27

Abstract:Memory is the foundation of all human activities; without memory, it would be nearly impossible for people to perform any task in daily life. With the development of Large Language Models (LLMs), their language capabilities are becoming increasingly comparable to those of humans. But do LLMs have memory? Based on current performance, LLMs do appear to exhibit memory. So, what is the underlying mechanism of this memory? Previous research has lacked a deep exploration of LLMs' memory capabilities and the underlying theory. In this paper, we use Universal Approximation Theorem (UAT) to explain the memory mechanism in LLMs. We also conduct experiments to verify the memory capabilities of various LLMs, proposing a new method to assess their abilities based on these memory ability. We argue that LLM memory operates like Schrödinger's memory, meaning that it only becomes observable when a specific memory is queried. We can only determine if the model retains a memory based on its output in response to the query; otherwise, it remains indeterminate. Finally, we expand on this concept by comparing the memory capabilities of the human brain and LLMs, highlighting the similarities and differences in their operational mechanisms.

Computation and Language
Classification and Generation of real-world data with an Associative Memory Model

Rodrigo Simas,Luis Sa-Couto,Andreas Wichert

DOI: https://doi.org/10.1016/j.neucom.2023.126514

2023-07-13

Abstract:Drawing from memory the face of a friend you have not seen in years is a difficult task. However, if you happen to cross paths, you would easily recognize each other. The biological memory is equipped with an impressive compression algorithm that can store the essential, and then infer the details to match perception. The Willshaw Memory is a simple abstract model for cortical computations which implements mechanisms of biological memories. Using our recently proposed sparse coding prescription for visual patterns, this model can store and retrieve an impressive amount of real-world data in a fault-tolerant manner. In this paper, we extend the capabilities of the basic Associative Memory Model by using a Multiple-Modality framework. In this setting, the memory stores several modalities (e.g., visual, or textual) of each pattern simultaneously. After training, the memory can be used to infer missing modalities when just a subset is perceived. Using a simple encoder-memory-decoder architecture, and a newly proposed iterative retrieval algorithm for the Willshaw Model, we perform experiments on the MNIST dataset. By storing both the images and labels as modalities, a single Memory can be used not only to retrieve and complete patterns but also to classify and generate new ones. We further discuss how this model could be used for other learning tasks, thus serving as a biologically-inspired framework for learning.

Neural and Evolutionary Computing,Artificial Intelligence,Machine Learning
Memory and attention in deep learning

Hung Le

DOI: https://doi.org/10.48550/arXiv.2107.01390

2021-07-03

Abstract:Intelligence necessitates memory. Without memory, humans fail to perform various nontrivial tasks such as reading novels, playing games or solving maths. As the ultimate goal of machine learning is to derive intelligent systems that learn and act automatically just like human, memory construction for machine is inevitable. Artificial neural networks model neurons and synapses in the brain by interconnecting computational units via weights, which is a typical class of machine learning algorithms that resembles memory structure. Their descendants with more complicated modeling techniques (a.k.a deep learning) have been successfully applied to many practical problems and demonstrated the importance of memory in the learning process of machinery systems. Recent progresses on modeling memory in deep learning have revolved around external memory constructions, which are highly inspired by computational Turing models and biological neuronal systems. Attention mechanisms are derived to support acquisition and retention operations on the external memory. Despite the lack of theoretical foundations, these approaches have shown promises to help machinery systems reach a higher level of intelligence. The aim of this thesis is to advance the understanding on memory and attention in deep learning. Its contributions include: (i) presenting a collection of taxonomies for memory, (ii) constructing new memory-augmented neural networks (MANNs) that support multiple control and memory units, (iii) introducing variability via memory in sequential generative models, (iv) searching for optimal writing operations to maximise the memorisation capacity in slot-based memory networks, and (v) simulating the Universal Turing Machine via Neural Stored-program Memory-a new kind of external memory for neural networks.

Machine Learning,Artificial Intelligence

A model of memory, learning and recognition

How the Brain Formulates Memory: A Spatio-Temporal Model Research Frontier.

Brain Inspired Cognitive System for Learning and Memory

A Formal Approach to Modeling the Memory of a Living Organism

A Neural Dynamic Model based on Activation Diffusion and a Micro-Explanation for Cognitive Operations

A Model of the Hippocampal–cortical Memory System

A Hierarchically Organized Memory Model With Temporal Population Coding

Discrete-state and continuous models of recognition memory: Testing core properties under minimal assumptions.

Introducing Memory and Association Mechanism into a Biologically Inspired Visual Model

Memory networks

A Cognitive Model Based Framework and Multi-layer Storage Architecture for Associative Memory

A New Pattern Associative Memory Model for Image Recognition Based on Hebb Rules and Dot Product

A critical comparison of discrete-state and continuous models of recognition memory: Implications for recognition and beyond.

Simple Models of Plant Learning and Memory

Hippocampal CA3–CA1 Synaptic Network Model of Memory

A Model of the Interaction Between Mood and Memory

A Machine with Short-Term, Episodic, and Semantic Memory Systems

A Full-Neuron Memory Model Designed for Neuromorphic Systems

Schrodinger's Memory: Large Language Models

Classification and Generation of real-world data with an Associative Memory Model

Memory and attention in deep learning