Abstract:Self-supervised learning of deep neural networks has become a prevalent paradigm for learning representations that transfer to a variety of downstream tasks. Similar to proposed models of the ventral stream of biological vision, it is observed that these networks lead to a separation of category manifolds in the representations of the penultimate layer. Although this observation matches the manifold hypothesis of representation learning, current self-supervised approaches are limited in their ability to explicitly model this manifold. Indeed, current approaches often only apply augmentations from a pre-specified set of "positive pairs" during learning. In this work, we propose a contrastive learning approach that directly models the latent manifold using Lie group operators parameterized by coefficients with a sparsity-promoting prior. A variational distribution over these coefficients provides a generative model of the manifold, with samples which provide feature augmentations applicable both during contrastive training and downstream tasks. Additionally, learned coefficient distributions provide a quantification of which transformations are most likely at each point on the manifold while preserving identity. We demonstrate benefits in self-supervised benchmarks for image datasets, as well as a downstream semi-supervised task. In the former case, we demonstrate that the proposed methods can effectively apply manifold feature augmentations and improve learning both with and without a projection head. In the latter case, we demonstrate that feature augmentations sampled from learned Lie group operators can improve classification performance when using few labels.

Self-Supervised Lie Algebra Representation Learning via Optimal Canonical Metric

Consistent Representation Learning for High Dimensional Data Analysis

Auto-Encoding Transformations in Reparameterized Lie Groups for Unsupervised Learning.

Learning Where to Learn in Cross-View Self-Supervised Learning

Manifold Contrastive Learning with Variational Lie Group Operators

Using Representation Expressiveness and Learnability to Evaluate Self-Supervised Learning Methods

On the Discriminability of Self-Supervised Representation Learning

Discriminative Subspace Learning With Adaptive Graph Regularization

Lie Group Manifold Analysis: an Unsupervised Domain Adaptation Approach for Image Classification

Learning Common Rationale to Improve Self-Supervised Representation for Fine-Grained Visual Recognition Problems

AETv2: AutoEncoding Transformations for Self-Supervised Representation Learning by Minimizing Geodesic Distances in Lie Groups

Semi-supervised Low-Rank Representation for Image Classification.

Self-Supervised Logit Adjustment

Enhanced low-rank representation via sparse manifold adaption for semi-supervised learning

Self-Supervised Representation Learning with Meta Comprehensive Regularization

Unsupervised Visual Representation Learning by Graph-Based Consistent Constraints.

Occam's Razor for Self Supervised Learning: What is Sufficient to Learn Good Representations?

Self-labelling via simultaneous clustering and representation learning

Regularized Semi-Supervised Latent Dirichlet Allocation for Visual Concept Learning

Align Representations with Base: A New Approach to Self-Supervised Learning

Sparse Representation Classifier Guided Grassmann Reconstruction Metric Learning With Applications to Image Set Analysis