Abstract:We introduce a conditional generative model for learning to disentangle the hidden factors of variation within a set of labeled observations, and separate them into complementary codes. One code summarizes the specified factors of variation associated with the labels. The other summarizes the remaining unspecified variability. During training, the only available source of supervision comes from our ability to distinguish among different observations belonging to the same class. Examples of such observations include images of a set of labeled objects captured at different viewpoints, or recordings of set of speakers dictating multiple phrases. In both instances, the intra-class diversity is the source of the unspecified factors of variation: each object is observed at multiple viewpoints, and each speaker dictates multiple phrases. Learning to disentangle the specified factors from the unspecified ones becomes easier when strong supervision is possible. Suppose that during training, we have access to pairs of images, where each pair shows two different objects captured from the same viewpoint. This source of alignment allows us to solve our task using existing methods. However, labels for the unspecified factors are usually unavailable in realistic scenarios where data acquisition is not strictly controlled. We address the problem of disentaglement in this more general setting by combining deep convolutional autoencoders with a form of adversarial training. Both factors of variation are implicitly captured in the organization of the learned embedding space, and can be used for solving single-image analogies. Experimental results on synthetic and real datasets show that the proposed method is capable of generalizing to unseen classes and intra-class variabilities.

Cultivating DNN Diversity for Large Scale Video Labelling

Aggregating Frame-level Features for Large-Scale Video Classification

Non-local NetVLAD Encoding for Video Classification

Beyond Instructional Videos: Probing for More Diverse Visual-Textual Grounding on YouTube

MOD: A Deep Mixture Model with Online Knowledge Distillation for Large Scale Video Temporal Concept Localization

On the Diversity of Synthetic Data and its Impact on Training Large Language Models

On Diversity in Discriminative Neural Networks

Disentangling Factors of Variation in Deep Representations Using Adversarial Training.

The Monkeytyping Solution to the YouTube-8M Video Understanding Challenge.

Curriculum Multi-Negative Augmentation for Debiased Video Grounding.

Beyond Scale: The Diversity Coefficient as a Data Quality Metric for Variability in Natural Language Data

Hierarchical Deep Recurrent Architecture for Video Understanding

UTS submission to Google YouTube-8M Challenge 2017.

T2Vid: Translating Long Text into Multi-Image is the Catalyst for Video-LLMs

Deep Multimodal Learning: An Effective Method for Video Classification

Diversity Boosted Learning for Domain Generalization with Large Number of Domains

Diversify and Conquer: Diversity-Centric Data Selection with Iterative Refinement

Scalable Ensemble Diversification for OOD Generalization and Detection

Exploring Resiliency to Natural Image Corruptions in Deep Learning using Design Diversity

Efficient Diversity-Driven Ensemble for Deep Neural Networks