Abstract:Unsupervised representation learning aims at finding methods that learn representations from data without annotation-based signals. Abstaining from annotations not only leads to economic benefits but may - and to some extent already does - result in advantages regarding the representation's structure, robustness, and generalizability to different tasks. In the long run, unsupervised methods are expected to surpass their supervised counterparts due to the reduction of human intervention and the inherently more general setup that does not bias the optimization towards an objective originating from specific annotation-based signals. While major advantages of unsupervised representation learning have been recently observed in natural language processing, supervised methods still dominate in vision domains for most tasks. In this dissertation, we contribute to the field of unsupervised (visual) representation learning from three perspectives: (i) Learning representations: We design unsupervised, backpropagation-free Convolutional Self-Organizing Neural Networks (CSNNs) that utilize self-organization- and Hebbian-based learning rules to learn convolutional kernels and masks to achieve deeper backpropagation-free models. (ii) Evaluating representations: We build upon the widely used (non-)linear evaluation protocol to define pretext- and target-objective-independent metrics for measuring and investigating the objective function mismatch between various unsupervised pretext tasks and target tasks. (iii) Transferring representations: We contribute CARLANE, the first 3-way sim-to-real domain adaptation benchmark for 2D lane detection, and a method based on prototypical self-supervised learning. Finally, we contribute a content-consistent unpaired image-to-image translation method that utilizes masks, global and local discriminators, and similarity sampling to mitigate content inconsistencies.

Unsupervised Visual Representation Learning by Context Prediction

Learning Visual Context for Group Activity Recognition.

Weakly-Supervised Spatial Context Networks.

Self-Supervised Graph Representation Learning via Global Context Prediction

Spatially-Aware Context Neural Networks.

Unsupervised Visual Representation Learning by Graph-Based Consistent Constraints.

Towards Unsupervised Representation Learning: Learning, Evaluating and Transferring Visual Representations

Discovering objects and their location in videos using spatial-temporal context words

Sparse Coding of Visual Context

Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles

Contextual associations represented both in neural networks and human behavior

Self-supervised Learning of Contextualized Local Visual Embeddings

Exploiting Contextual Uncertainty of Visual Data for Efficient Training of Deep Models

Non-parametric Contextual Relationship Learning for Semantic Video Object Segmentation

Unsupervised Object Discovery Via Object-Centric Representation

Learning 3D object-centric representation through prediction

Efficient Self-supervised Learning with Contextualized Target Representations for Vision, Speech and Language

Temporally-Embedded Self-Supervised Video Representation Learning

Unsupervised Object-Centric Learning from Multiple Unspecified Viewpoints

Context-LGM: Leveraging Object-Context Relation for Context-Aware Object Recognition

Unsupervised Discovery of Object-Centric Neural Fields