Abstract:Background: Image memorability refers to the phenomenon where certain images are more likely to be remembered than others. It is a quantifiable and intrinsic image attribute, defined as the likelihood of being remembered upon a single exposure. Despite advances in understanding human visual perception and memory, it is unclear what features contribute to an image's memorability. To address this question, we propose a deep learning-based computational modeling approach. Methods: We modeled the subjective experience of visual memorability using an autoencoder based on VGG16 Convolutional Neural Networks (CNNs). The model was trained on images for one epoch, to simulate the single-exposure condition used in human memory tests. We investigated the relationship between memorability and reconstruction error, assessed latent space representations distinctiveness, and developed a Gated Recurrent Unit (GRU) model to predict memorability likelihood. Interpretability analysis was conducted to identify key image characteristics contributing to memorability. Results: Our results demonstrate a significant correlation between the images memorability score and autoencoder's reconstruction error, and the robust predictive performance of its latent representations. Distinctiveness in these representations correlated significantly with memorability. Additionally, certain visual characteristics, such as strong contrasts, distinctive objects, and prominent foreground elements were among the features contributing to image memorability in our model. Conclusions: Images with unique features that challenge the autoencoder's capacity are inherently more memorable. Moreover, these memorable images are distinct from others the model has encountered, and the latent space of the encoder contains features predictive of memorability.

Localizing Memorization in SSL Vision Encoders

Memorization in Self-Supervised Learning Improves Downstream Generalization

Localizing Paragraph Memorization in Language Models

Do SSL Models Have Déjà Vu? A Case of Unintended Memorization in Self-supervised Learning

Generalisation First, Memorisation Second? Memorisation Localisation for Natural Language Classification Tasks

Finding NeMo: Localizing Neurons Responsible For Memorization in Diffusion Models

Déjà Vu Memorization in Vision-Language Models

SoK: Memorization in General-Purpose Large Language Models

Embedding Space Selection for Detecting Memorization and Fingerprinting in Generative Models

Learning Where to Learn in Cross-View Self-Supervised Learning

A Multi-Perspective Analysis of Memorization in Large Language Models

Memorization in deep learning: A survey

Detecting Memorization in Large Language Models

Undesirable Memorization in Large Language Models: A Survey

Unlocking Memorization in Large Language Models with Dynamic Soft Prompting

Modeling Visual Memorability Assessment with Autoencoders Reveals Characteristics of Memorable Images

Exploring Local Memorization in Diffusion Models via Bright Ending Attention

On the Generalization and Causal Explanation in Self-Supervised Learning

Detecting Unintended Memorization in Language-Model-Fused ASR

CNN with large memory layers