Abstract:The currently leading artificial neural network (ANN) models of the visual ventral stream -- which are derived from a combination of performance optimization and robustification methods -- have demonstrated a remarkable degree of behavioral alignment with humans on visual categorization tasks. Extending upon previous work, we show that not only can these models guide image perturbations that change the induced human category percepts, but they also can enhance human ability to accurately report the original ground truth. Furthermore, we find that the same models can also be used out-of-the-box to predict the proportion of correct human responses to individual images, providing a simple, human-aligned estimator of the relative difficulty of each image. Motivated by these observations, we propose to augment visual learning in humans in a way that improves human categorization accuracy at test time. Our learning augmentation approach consists of (i) selecting images based on their model-estimated recognition difficulty, and (ii) using image perturbations that aid recognition for novice learners. We find that combining these model-based strategies gives rise to test-time categorization accuracy gains of 33-72% relative to control subjects without these interventions, despite using the same number of training feedback trials. Surprisingly, beyond the accuracy gain, the training time for the augmented learning group was also shorter by 20-23%. We demonstrate the efficacy of our approach in a fine-grained categorization task with natural images, as well as tasks in two clinically relevant image domains -- histology and dermoscopy -- where visual learning is notoriously challenging. To the best of our knowledge, this is the first application of ANNs to increase visual learning performance in humans by enhancing category-specific features.

A Latent Variable Augmentation Method for Image Categorization with Insufficient Training Samples

Explicitly Learning Augmentation Invariance for Image Classification by Consistent Augmentation

Data-Efficient Augmentation for Training Neural Networks

Augmentation Invariant Training

Enhancing Generalization of Universal Adversarial Perturbation Through Gradient Aggregation

Semantic feature augmentation for fine-grained visual categorization with few-sample training

Towards Composable Distributions of Latent Space Augmentations

Category-Oriented Adversarial Data Augmentation Via Statistic Similarity for Satellite Images

Soft Augmentation for Image Classification

Learning Optimal Data Augmentation Policies via Bayesian Optimization for Image Classification Tasks

MaskCOV: A random mask covariance network for ultra-fine-grained visual categorization

NormAUG: Normalization-Guided Augmentation for Domain Generalization

Self-Paced Video Data Augmentation by Generative Adversarial Networks with Insufficient Samples.

LatentAugment: Dynamically Optimized Latent Probabilities of Data Augmentation

L-WISE: Boosting Human Image Category Learning Through Model-Based Image Selection And Enhancement

CVAE-GAN: Fine-Grained Image Generation through Asymmetric Training

Low-Sample Image Classification Based on Intrinsic Consistency Loss and Uncertainty Weighting Method

LGV: Boosting Adversarial Example Transferability from Large Geometric Vicinity

A Galaxy Image Augmentation Method Based on Few-shot Learning and Generative Adversarial Networks

Improving Generalization of Convolutional Neural Network Through Contrastive Augmentation