Abstract:Despite decades of research, much is still unknown about the computations carried out in the human face processing network. Recently, deep networks have been proposed as a computational account of human visual processing, but while they provide a good match to neural data throughout visual cortex, they lack interpretability. We introduce a method for interpreting brain activity using a new class of deep generative models, disentangled representation learning models, which learn a low-dimensional latent space that "disentangles" different semantically meaningful dimensions of faces, such as rotation, lighting, or hairstyle, in an unsupervised manner by enforcing statistical independence between dimensions. We find that the majority of our model's learned latent dimensions are interpretable by human raters. Further, these latent dimensions serve as a good encoding model for human fMRI data. We next investigate the representation of different latent dimensions across face-selective voxels. We find that low- and high-level face features are represented in posterior and anterior face-selective regions, respectively, corroborating prior models of human face recognition. Interestingly, though, we find identity-relevant and irrelevant face features across the face processing network. Finally, we provide new insight into the few "entangled" (uninterpretable) dimensions in our model by showing that they match responses in the ventral stream and carry information about facial identity. Disentangled face encoding models provide an exciting alternative to standard "black box" deep learning approaches for modeling and interpreting human brain data. We use a class of interpretable deep neural network models, disentangled variational autoencoders (dVAEs), to analyze human fMRI data. We find that a dVAE learns human interpretable dimensions of faces, such as lighting, expression, and hairstyle, and provides as good a match to human fMRI data as matched, non-disentangled models. Our disentangled encoding approach allows us to map different disentangled features to ROI and voxel activity. A decoding analysis confirms that the model separates identity relevant and irrelevant information and reveals that the remaining entangled dimensions contain identity-relevant information. Together these results highlight the use of disentangled models for more interpretable fMRI encoding than standard deep learning models.

Learning Disentangled Representations via Independent Subspaces

Realistic Face Reenactment Via Self-Supervised Disentangling of Identity and Pose

Facial Landmark Disentangled Network with Variational Autoencoder

Controllable Face Image Editing in a Disentanglement Way

Learning Distribution Independent Latent Representation for 3D Face Disentanglement.

Disentangled Representation Learning for Multiple Attributes Preserving Face Deidentification

Disentangling Factors of Variation in Deep Representations Using Adversarial Training.

Neural Face Editing with Intrinsic Image Disentangling

Disentanglement for Discriminative Visual Recognition

Toward a Controllable Disentanglement Network

Learning a Self-Expressive Network for Subspace Clustering

Learning Disentangled Representation for Robust Person Re-identification

Learning Disentangled Representations via Mutual Information Estimation

Improving the Reconstruction of Disentangled Representation Learners Via Multi-Stage Modelling.

Learning Controllable Disentangled Representations with Decorrelation Regularization

Learning Disentangled Discrete Representations

APGVAE: Adaptive Disentangled Representation Learning with the Graph-Based Structure Information

Facial Expression Recognition Using Disentangled Adversarial Learning

Disentangled deep generative models reveal coding principles of the human face processing network

Disentangled Representations for Short-Term and Long-Term Person Re-Identification

Disentangled Representations in Neural Models