Associative Memories in the Feature Space

Tommaso Salvatori,Beren Millidge,Yuhang Song,Rafal Bogacz,Thomas Lukasiewicz

2024-02-17

Abstract:An autoassociative memory model is a function that, given a set of data points, takes as input an arbitrary vector and outputs the most similar data point from the memorized set. However, popular memory models fail to retrieve images even when the corruption is mild and easy to detect for a human evaluator. This is because similarities are evaluated in the raw pixel space, which does not contain any semantic information about the images. This problem can be easily solved by computing \emph{similarities} in an embedding space instead of the pixel space. We show that an effective way of computing such embeddings is via a network pretrained with a contrastive loss. As the dimension of embedding spaces is often significantly smaller than the pixel space, we also have a faster computation of similarity scores. We test this method on complex datasets such as CIFAR10 and STL10. An additional drawback of current models is the need of storing the whole dataset in the pixel space, which is often extremely large. We relax this condition and propose a class of memory models that only stores low-dimensional semantic embeddings, and uses them to retrieve similar, but not identical, memories. We demonstrate a proof of concept of this method on a simple task on the MNIST dataset.

Machine Learning,Computer Vision and Pattern Recognition

What problem does this paper attempt to address?

The main problem that this paper attempts to solve is the defects of current associative memory models when processing image data, which are specifically manifested in the following aspects:

1. **Poor image retrieval ability**: Existing associative memory models (such as Hopfield networks) have difficulty in correctly retrieving the original image when facing image data, even if the image is only slightly damaged or changed. This is because these models use the original pixel space when calculating similarity, and this space lacks the semantic information of the image, resulting in the model being insensitive to the "semantic" features in the image.

2. **Low computational efficiency**: Since similarity calculations need to be carried out in the high - dimensional pixel space, this not only increases the computational complexity but also leads to high memory consumption. Especially when dealing with large - scale data sets, storage and retrieval efficiency become a bottleneck.

3. **Low storage efficiency**: Existing associative memory models usually need to store the original pixel information of the entire data set, which is very uneconomical for large - scale data sets. This not only occupies a large amount of storage space but also limits the scalability of the model.

To solve the above problems, the paper proposes the following methods:

- **Similarity calculation in the embedding space**: Map the image data to a low - dimensional embedding space and then calculate the similarity within this space. In this way, pre - trained neural networks (such as ResNet18 and ResNet50) can be used to extract the semantic features of the image, thereby increasing the model's sensitivity to image semantics.

- **Network pre - trained with contrastive loss**: The paper uses a neural network pre - trained with contrastive loss as a feature mapping function. This method can effectively map different versions of the same category to close positions and map images of different categories to far - away positions.

- **Low - dimensional embedding storage**: To improve storage efficiency, the paper proposes a method of only storing low - dimensional embedding vectors and reconstructs the image from the embedding space through a generative network. In this way, the storage requirements can be significantly reduced while ensuring the retrieval quality.

Through these methods, the paper aims to construct a more efficient and robust associative memory model that can exhibit better performance when processing complex image data.