Abstract:The human brain exhibits a strong ability to spontaneously associate different visual attributes of the same or similar visual scene, such as associating sketches and graffiti with real-world visual objects, usually without supervising information. In contrast, in the field of artificial intelligence, controllable generation methods like ControlNet heavily rely on annotated training datasets such as depth maps, semantic segmentation maps, and poses, which limits the method's scalability. Inspired by the neural mechanisms that may contribute to the brain's associative power, specifically the cortical modularization and hippocampal pattern completion, here we propose a self-supervised controllable generation (SCG) framework. Firstly, we introduce an equivariant constraint to promote inter-module independence and intra-module correlation in a modular autoencoder network, thereby achieving functional specialization. Subsequently, based on these specialized modules, we employ a self-supervised pattern completion approach for controllable generation training. Experimental results demonstrate that the proposed modular autoencoder effectively achieves functional specialization, including the modular processing of color, brightness, and edge detection, and exhibits brain-like features including orientation selectivity, color antagonism, and center-surround receptive fields. Through self-supervised training, associative generation capabilities spontaneously emerge in SCG, demonstrating excellent generalization ability to various tasks such as associative generation on painting, sketches, and ancient graffiti. Compared to the previous representative method ControlNet, our proposed approach not only demonstrates superior robustness in more challenging high-noise scenarios but also possesses more promising scalability potential due to its self-supervised manner.

MultiselfGAN: A Self-Guiding Neural Architecture Search Method for Generative Adversarial Networks with Multicontrollers

Distributed GAN: Toward a Faster Reinforcement-Learning-Based Architecture Search

Determining Learning Direction Via Multi-Controller Model for Stably Searching Generative Adversarial Networks

EfficientAutoGAN: Predicting the Rewards in Reinforcement-Based Neural Architecture Search for Generative Adversarial Networks.

Off-Policy Reinforcement Learning for Efficient and Effective GAN Architecture Search

AutoGAN: Neural Architecture Search for Generative Adversarial Networks

Spatial Steerability of GANs via Self-Supervision from Discriminator

AdversarialNAS: Adversarial Neural Architecture Search for GANs.

Toward Gradient Bandit-Based Selection of Candidate Architectures in AutoGAN

Generative Adversarial Neural Architecture Search

AutoGAN-DSP: Stabilizing GAN architecture search with deterministic score predictors

Coarse-to-Fine Searching for Efficient Generative Adversarial Networks.

Autonomously and Simultaneously Refining Deep Neural Network Parameters by a Bi-Generative Adversarial Network Aided Genetic Algorithm

Controlling the Latent Space of GANs through Reinforcement Learning: A Case Study on Task-based Image-to-Image Translation

Learning from Pattern Completion: Self-supervised Controllable Generation

MCL-GAN: Generative Adversarial Networks with Multiple Specialized Discriminators

Self-supervised learning with consistency loss for improving GANs

A neural architecture generator for efficient search space

Generative Adversarial Exploration for Reinforcement Learning

Searching Towards Class-Aware Generators for Conditional Generative Adversarial Networks

Multigoal Visual Navigation With Collision Avoidance via Deep Reinforcement Learning