Abstract:Background: Image memorability refers to the phenomenon where certain images are more likely to be remembered than others. It is a quantifiable and intrinsic image attribute, defined as the likelihood of being remembered upon a single exposure. Despite advances in understanding human visual perception and memory, it is unclear what features contribute to an image's memorability. To address this question, we propose a deep learning-based computational modeling approach. Methods: We modeled the subjective experience of visual memorability using an autoencoder based on VGG16 Convolutional Neural Networks (CNNs). The model was trained on images for one epoch, to simulate the single-exposure condition used in human memory tests. We investigated the relationship between memorability and reconstruction error, assessed latent space representations distinctiveness, and developed a Gated Recurrent Unit (GRU) model to predict memorability likelihood. Interpretability analysis was conducted to identify key image characteristics contributing to memorability. Results: Our results demonstrate a significant correlation between the images memorability score and autoencoder's reconstruction error, and the robust predictive performance of its latent representations. Distinctiveness in these representations correlated significantly with memorability. Additionally, certain visual characteristics, such as strong contrasts, distinctive objects, and prominent foreground elements were among the features contributing to image memorability in our model. Conclusions: Images with unique features that challenge the autoencoder's capacity are inherently more memorable. Moreover, these memorable images are distinct from others the model has encountered, and the latent space of the encoder contains features predictive of memorability.

Predicting Visual Memory Schemas with Variational Autoencoders

Generating Memorable Images Based on Human Visual Memory Schemas

Defining Image Memorability using the Visual Memory Schema

Discrete Memory Addressing Variational Autoencoder for Visual Concept Learning

Variational Autoencoder: An Unsupervised Model for Modeling and Decoding fMRI Activity in Visual Cortex

Modeling Visual Memorability Assessment with Autoencoders Reveals Characteristics of Memorable Images

Sparse-Coding Variational Auto-Encoders

Visualization Assessment - A Machine Learning Approach.

Sparse-Coding Variational Autoencoders

Déjà Vu Memorization in Vision-Language Models

Fusing multimodal neuroimaging data with a variational autoencoder

Multi-Modal Latent Variables for Cross-Individual Primary Visual Cortex Modeling and Analysis

Scaling models of visual working memory to natural images

InVA: Integrative Variational Autoencoder for Harmonization of Multi-modal Neuroimaging Data

Visual Memory Neural Network for Artistic Graphic Design

Reconstructing seen image from brain activity by visually-guided cognitive representation and adversarial learning

Learning Audio-Visual Correlations from Variational Cross-Modal Generation

ViVA: Semi-Supervised Visualization Via Variational Autoencoders

Using a Vertical-Stream Variational Auto-Encoder to Generate Segment-Based Images and Its Biological Plausibility for Modelling the Visual Pathways.

Architecture Design for Variational Auto-Encoders

Visual Episodic Memory-based Exploration