Abstract:Unsupervised representation learning aims at finding methods that learn representations from data without annotation-based signals. Abstaining from annotations not only leads to economic benefits but may - and to some extent already does - result in advantages regarding the representation's structure, robustness, and generalizability to different tasks. In the long run, unsupervised methods are expected to surpass their supervised counterparts due to the reduction of human intervention and the inherently more general setup that does not bias the optimization towards an objective originating from specific annotation-based signals. While major advantages of unsupervised representation learning have been recently observed in natural language processing, supervised methods still dominate in vision domains for most tasks. In this dissertation, we contribute to the field of unsupervised (visual) representation learning from three perspectives: (i) Learning representations: We design unsupervised, backpropagation-free Convolutional Self-Organizing Neural Networks (CSNNs) that utilize self-organization- and Hebbian-based learning rules to learn convolutional kernels and masks to achieve deeper backpropagation-free models. (ii) Evaluating representations: We build upon the widely used (non-)linear evaluation protocol to define pretext- and target-objective-independent metrics for measuring and investigating the objective function mismatch between various unsupervised pretext tasks and target tasks. (iii) Transferring representations: We contribute CARLANE, the first 3-way sim-to-real domain adaptation benchmark for 2D lane detection, and a method based on prototypical self-supervised learning. Finally, we contribute a content-consistent unpaired image-to-image translation method that utilizes masks, global and local discriminators, and similarity sampling to mitigate content inconsistencies.

Enhancing 2D Representation Learning with a 3D Prior

Learning 3D Scene Priors with 2D Supervision

Three-Dimensional Reconstruction Pre-Training as a Prior to Improve Robustness to Adversarial Attacks and Spurious Correlation

3D estimation of single-view 2d images using shape priors and transfer learning

Improving 2D Feature Representations by 3D-Aware Fine-Tuning

Visual Reinforcement Learning with Self-Supervised 3D Representations

Unsupervised Learning of 3D Structure from Images

Self-Supervised 3D Representation Learning of Dressed Humans From Social Media Videos

Semi-supervised Single-view 3D Reconstruction via Multi Shape Prior Fusion Strategy and Self-Attention

Semantically-Guided Representation Learning for Self-Supervised Monocular Depth

Deep Optimized Priors for 3D Shape Modeling and Reconstruction

Semi-Supervised Single-View 3D Reconstruction Via Prototype Shape Priors

Towards Unsupervised Representation Learning: Learning, Evaluating and Transferring Visual Representations

Neural View Synthesis and Matching for Semi-Supervised Few-Shot Learning of 3D Pose

Temporally-Embedded Self-Supervised Video Representation Learning

View-to-Label: Multi-View Consistency for Self-Supervised 3D Object Detection

Self-supervised Secondary Landmark Detection via 3D Representation Learning

Monocular Depth Estimation Using Self-Supervised Learning with More Effective Geometric Constraints

FSD: Fast Self-Supervised Single RGB-D to Categorical 3D Objects

Self-Supervised Multi-View Learning via Auto-Encoding 3D Transformations