Abstract:This dissertation attempts to drive innovation in the field of generative modeling for computer vision, by exploring novel formulations of conditional generative models, and innovative applications in images, 3D animations, and video. Our research focuses on architectures that offer reversible transformations of noise and visual data, and the application of encoder-decoder architectures for generative tasks and 3D content manipulation. In all instances, we incorporate conditional information to enhance the synthesis of visual data, improving the efficiency of the generation process as well as the generated content. We introduce the use of Neural ODEs to model video dynamics using an encoder-decoder architecture, demonstrating their ability to predict future video frames despite being trained solely to reconstruct current frames. Next, we propose a conditional variant of continuous normalizing flows that enables higher-resolution image generation based on lower-resolution input, achieving comparable image quality while reducing parameters and training time. Our next contribution presents a pipeline that takes human images as input, automatically aligns a user-specified 3D character with the pose of the human, and facilitates pose editing based on partial inputs. Next, we derive the relevant mathematical details for denoising diffusion models that use non-isotropic Gaussian processes, and show comparable generation quality. Finally, we devise a novel denoising diffusion framework capable of solving all three video tasks of prediction, generation, and interpolation. We perform ablation studies, and show SOTA results on multiple datasets. Our contributions are published articles at peer-reviewed venues. Overall, our research aims to make a meaningful contribution to the pursuit of more efficient and flexible generative models, with the potential to shape the future of computer vision.

Conditional Idempotent Generative Networks

Statistics Enhancement Generative Adversarial Networks for Diverse Conditional Image Synthesis

Conditional Generative Adversarial Nets

Attack Deterministic Conditional Image Generative Models for Diverse and Controllable Generation

Conditional Information Gain Networks

Ways of Conditioning Generative Adversarial Networks

Idempotent Generative Network

Omni-GAN: on the Secrets of Cgans and Beyond

IGAN: Inferent and Generative Adversarial Networks

Incremental Focal Loss GANs.

Condition-Aware Neural Network for Controlled Image Generation

Conditional computation in neural networks: principles and research trends

GANs Conditioning Methods: A Survey

Object-Centric Relational Representations for Image Generation

Conditional Image Generation with PixelCNN Decoders

Conditional Generative Modeling for Images, 3D Animations, and Video

Guiding GANs: How to control non-conditional pre-trained GANs for conditional image generation

CIPS-3D: A 3D-Aware Generator of GANs Based on Conditionally-Independent Pixel Synthesis

Network-to-Network Translation with Conditional Invertible Neural Networks

Conditional Image Synthesis With Auxiliary Classifier GANs

GIU-GANs: Global Information Utilization for Generative Adversarial Networks