Abstract:The human visual system can easily recognize object material categories and estimate surface properties such as glossiness and smoothness. A number of psychophysical and computational studies suggest that the material perception depends on global feature statistics of the entire image at multiple processing levels. Neural representations of such global features, which is independent of precise retinotopy, may be captured even by EEG that have low spatial resolution. To test this possibility, here we measured visual evoked potentials (VEPs) for 191 natural images consisting of 20 categories of materials. We then sought to classify material categories and surface properties from the VEPs, and to reconstruct the rich phenomenological appearance of materials themselves via neural representations of global features as estimated from the VEPs. As a result, we found that material categories were correctly classified by the VEPs even at latencies of 150 ms or less. The apparent surface properties were also significantly classified within 175 ms (lightness, colorfulness, and smoothness) and after 200 ms (glossiness, hardness, and heaviness). In a subsequent reverse-correlation analysis, we further found that the VEPs at these latencies are highly correlated with low- and high-level global feature statistics of the surface images; Portilla-Simoncelli texture statistics and style information in deep convolutional neural network (dCNN), indicating that neural activities about such global features are reflected in the VEPs that enabled successful classification of materials. To demonstrate this idea more directly, we trained deep generative models (MVAE models) that reconstruct the surface image itself from the VEPs via style information (gram matrix of the dCNN output). The model successfully reconstructed realistic surface images, a part of which were nearly indistinguishable from the original images. These findings suggest that the neural representation of statistical image features, which were formed at short latencies in the visual cortex and reflected even in EEG signals, not simply enable human visual system to recognize material categories and evaluate surface properties but provides the essential basis for rich and complex phenomenological qualities of natural surfaces.

Photorealistic Reconstruction of Visual Texture From EEG Signals

Rethinking Visual Reconstruction: Experience-Based Content Completion Guided by Visual Cues

Reconstructing Visual Stimulus Images from EEG Signals Based on Deep Visual Representation Model

Neural basis of perceptual surface qualities: Evidence from EEG decoding

Image classification and reconstruction from low-density EEG

Seeing through the Brain: Image Reconstruction of Visual Perception from Human Brain Signals

Deep image reconstruction from human brain activity

Visual Decoding and Reconstruction via EEG Embeddings with Guided Diffusion

Reconstruction of natural images from evoked brain activity with a dictionary-based invertible encoding procedure

Spurious reconstruction from brain activity

Brain decoding: toward real-time reconstruction of visual perception

Reconstruction of the External Stimuli from Brain Signals

DREAM: Visual Decoding from Reversing Human Visual System

EEG-Driven 3D Object Reconstruction with Color Consistency and Diffusion Prior

Reconstructing the Mind's Eye: fMRI-to-Image with Contrastive Learning and Diffusion Priors

Images Structure Reconstruction from Fmri by Unsupervised Learning Based on VAE

EEG2IMAGE: Image Reconstruction from EEG Brain Signals

Reconstructing Sources Location of Visual Color Cortex by the Task-Irrelevant Visual Stimuli Through Machine Learning Decoding

More Than Meets the Eye: Self-Supervised Depth Reconstruction From Brain Activity

Image Reconstruction from Electroencephalography Using Latent Diffusion

Decoding visual brain representations from electroencephalography through Knowledge Distillation and latent diffusion models