Abstract:Human neuroimaging studies have shown that the contents of episodic memories are represented in distributed patterns of neural activity. However, these studies have mostly been limited to decoding simple, unidimensional properties of stimuli. Semantic encoding models, in contrast, offer a means for characterizing the rich, multidimensional information that comprises episodic memories. Here, we extensively sampled four human fMRI subjects to build semantic encoding models and then applied these models to reconstruct content from natural scene images as they were viewed and recalled from memory. First, we found that multidimensional semantic information was successfully reconstructed from activity patterns across visual and lateral parietal cortices, both when viewing scenes and when recalling them from memory. Second, whereas visual cortical reconstructions were much more accurate when images were viewed versus recalled from memory, lateral parietal reconstructions were comparably accurate across visual perception and memory. Third, by applying natural language processing methods to verbal recall data, we showed that fMRI-based reconstructions reliably matched subjects’ verbal descriptions of their memories. In fact, reconstructions from ventral temporal cortex more closely matched subjects’ own verbal recall than other subjects’ verbal recall of the same images. Fourth, encoding models reliably transferred across subjects: memories were successfully reconstructed using encoding models trained on data from entirely independent subjects. Together, these findings provide evidence for successful reconstructions of multidimensional and idiosyncratic memory representations and highlight the differential sensitivity of visual cortical and lateral parietal regions to information derived from the external visual environment versus internally-generated memories. ### Competing Interest Statement The authors have declared no competing interest.

A Model of Semantic Completion in Generative Episodic Memory

A generative model of memory construction and consolidation

Few-shot Generation Via Recalling Brain-Inspired Episodic-Semantic Memory.

Consolidation of sequential experience into a deep generative network explains human memory, prediction and planning

MemoryGAN: GAN Generator As Heterogeneous Memory for Compositional Image Synthesis

Superposed Episodic and Semantic Memory via Sparse Distributed Representation

Associative Memories via Predictive Coding

Mapping multidimensional content representations to neural and behavioral expressions of episodic memory

How the Brain Formulates Memory: A Spatio-Temporal Model Research Frontier.

Spiking neural network-based computational modeling of episodic memory

Neural models of episodic memory

A hippocampus CA3 spiking neural network model for storage and retrieval of sequential memory

Episodic and associative memory from spatial scaffolds in the hippocampus

Contingency of semantic generalization on episodic specificity varies across development

Memory Encoding Model

Beyond the episodic–semantic continuum: the multidimensional model of mental representations

Imagery in the entropic associative memory

In search of dispersed memories: Generative diffusion models are associative memory networks

Semantic Neural Decoding via Cross-Modal Generation

A Unified Model of Spatial and Episodic Memory.